Why is it fast
Search replica uses jackc’s pgconn & pgtype libraries, which can decode postgres data quite efficiently. Most types in pgtype, implement JSON marshalling, making this step even more efficient.
Uses binary format for initial reindexing and streaming (PG 14+)
Both COPY TO
and Streamed messaged are processed one by one, reusing memory. Keeping memory requirements low.
All updates are combined into bulk buffer, which is flushed by predefined intervals.
No updates - no requests.
In case, if SearchReplica was in idle state, and message arrives, it would be forwarded immediately.
- No additional DB requests
- No dependencies, queues, locks, network delay.
Just fetch => decode => encode => push the data.
Complex features, like inlining, use scripted update. It’s relatively heavy operation, but way cheaper than alternatives and does not distrupt streaming.