to send a out-of-range integer as a 2-byte value

Source: Internet
Author: User
Tags mongodb postgresql postgresql client

The newest feature in the Software I ' m developing to Just software is a workstream, which are our aggregate name for MicroB Log and activity stream functionality. Since we ' re using a relational DB we ' ve modeled this module in the way then every WorkStream message be attached to a user ' s stream in a simple m:n relation manner. To reduce the number of INSERT statements the list of streams are assembled in the business service and then The database service which does sth. Like this:

INSERT into WorkStream (MsgId, UserID) VALUES (5, 1), Values (5, 2), ...;

This is however can result in a huge amount of VALUES (...) parameters when the platform has users. It didn ' t occur as a problem to me when implementing it; PostgreSQL can handle large data, no? The IT ' s at least limited. When to use a JDBC prepared statement then the number of parameters has a hard limit:32767. If you don ' t obey this limit you'll be something like this: to send an out-of-range integer as a 2-byte value:40000

Not a very concrete message, right? When I-I-exception I was kind of baffled and disaffected. But after narrowing, the problem in a debug session and looking at the PostgreSQL JDBC driver ' s source code the cause was obvious:the PostgreSQL client/backend protocol dictates this number of parameters being send from the client to the Postgres backend as a 2 byte integer (Aaah, now the above message actually makes). You'll find details of the protocol this brave (my 2-byte friend are defined in the Parse message).

All right, got the bugger, what's the solution now? Easy:just split the parameter list up to chunks of say 30.000 parameters per INSERT statement.


Problem: When the JDBC driver for PG prepared statement parameter set, a sending size on the client side is limited to 2-byte.

Workaround: The current version maximum limit of 50132, recommended each split in 50,000 parameters to execute.

Alibaba Cloud Hot Products

Elastic Compute Service (ECS) Dedicated Host (DDH) ApsaraDB RDS for MySQL (RDS) ApsaraDB for PolarDB(PolarDB) AnalyticDB for PostgreSQL (ADB for PG)
AnalyticDB for MySQL(ADB for MySQL) Data Transmission Service (DTS) Server Load Balancer (SLB) Global Accelerator (GA) Cloud Enterprise Network (CEN)
Object Storage Service (OSS) Content Delivery Network (CDN) Short Message Service (SMS) Container Service for Kubernetes (ACK) Data Lake Analytics (DLA)

ApsaraDB for Redis (Redis)

ApsaraDB for MongoDB (MongoDB) NAT Gateway VPN Gateway Cloud Firewall
Anti-DDoS Web Application Firewall (WAF) Log Service DataWorks MaxCompute
Elastic MapReduce (EMR) Elasticsearch

Alibaba Cloud Free Trail

Related Article

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: and provide relevant evidence. A staff member will contact you within 5 working days.

A Free Trial That Lets You Build Big!

Start building with 50+ products and up to 12 months usage for Elastic Compute Service

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.