Incident brief
Invalid byte sequence for encoding
Interpreting bytes as text in a given encoding fails when the bytes are not valid there; 0xff is never a valid standalone UTF-8 byte.
In 10 seconds
- What
- Invalid byte sequence for encoding
- What triggers it
- Call convert_from(bytea, encoding) where the bytes are not valid in the stated encoding.
- The fix
- Feed bytes that are actually valid in the declared encoding.
- Proof
- Reproduced on PostgreSQL 18.4 → A single SELECT reproduces SQLSTATE 22021; the byte 0xff is rejected as invalid UTF-8.
The fix
What to do right now
The immediate, application-level response to this error.
- Feed bytes that are actually valid in the declared encoding.
- If the true encoding differs, decode with the correct one (for example LATIN1) instead of UTF8, or sanitize the input first.
-- Decode only bytes that are valid in the target encoding.
SELECT convert_from('\x41'::bytea, 'UTF8');Diagnose
See it live on the server
Run these against the affected instance to confirm the diagnosis before you act.
Standard triage — not specific to this error
These are canonical PostgreSQL system-catalog queries, shown as SQL to run. No sample output is attached because this is general triage, not a captured lab transcript.This SQLSTATE does not have an error-specific live snapshot yet. These are the canonical system-catalog queries you run against the affected server to see the problem in real time — standard triage, not a reproduced transcript.
What is running right now
Active backends, how long each has been running, and what it is waiting on.
SELECT pid,
state,
wait_event_type,
wait_event,
now() - query_start AS running_for,
left(query, 80) AS query
FROM pg_stat_activity
WHERE state <> 'idle'
AND pid <> pg_backend_pid()
ORDER BY running_for DESC NULLS LAST;Who is blocking whom
Turn raw blocking PIDs into the actual queries on both sides of the wait.
SELECT blocked.pid AS blocked_pid,
blocked.query AS blocked_query,
blocking.pid AS blocking_pid,
blocking.query AS blocking_query
FROM pg_stat_activity AS blocked
JOIN LATERAL unnest(pg_blocking_pids(blocked.pid)) AS b(pid) ON true
JOIN pg_stat_activity AS blocking ON blocking.pid = b.pid
WHERE cardinality(pg_blocking_pids(blocked.pid)) > 0;Locks that are still waiting
Every lock a backend has requested but not yet been granted.
SELECT l.pid,
l.locktype,
l.mode,
l.granted,
COALESCE(c.relname, l.transactionid::text) AS object
FROM pg_locks l
LEFT JOIN pg_class c ON c.oid = l.relation
WHERE NOT l.granted
ORDER BY l.pid;Why it happens
What PostgreSQL is telling you
The mechanism behind the error, grounded in the official manual — not paraphrased.
PostgreSQL 18 Documentation — Appendix A. PostgreSQL Error Codes (Table A.1, Class 22 — Data Exception)
22021 → character_not_in_repertoireRead the full section on postgresql.org →
Decoding an invalid byte as UTF-8
0xff cannot appear in a valid UTF-8 sequence, so PostgreSQL raises 'invalid byte sequence for encoding UTF8: 0xff'.The session continues normally
The failing statement ran outside a transaction block, so nothing was left in a bad state — the next statement in the same connection runs normally.Reproduce & verify
A real, single-session PostgreSQL reproduction
A literal transcript of SQL run against a live PostgreSQL instance in an isolated lab — the commands below are exactly what was executed.
- 1Call convert_from(bytea, encoding) where the bytes are not valid in the stated encoding.
- 2PostgreSQL decodes the bytes and reaches 0xff, which cannot begin or continue a UTF-8 sequence.
- 3The decode aborts with SQLSTATE 22021 and 'invalid byte sequence for encoding UTF8: 0xff'.
A byte column captured from a mixed-encoding upstream was decoded as UTF-8 and failed on the first non-UTF-8 byte.
-- The error is self-contained in one statement; no schema is required.
SELECT 'no schema needed' AS setup_note;SELECT convert_from('\xff'::bytea, 'UTF8');SELECT 'ok' AS session_after_error;What PostgreSQL actually returned
setup_note
------------------
no schema needed
(1 row)ERROR: invalid byte sequence for encoding "UTF8": 0xff session_after_error
---------------------
ok
(1 row)A byte that is valid in the encoding decodes to its character without error.
We decode a valid ASCII byte as UTF-8 to show the resulting text.
Without this
Before: 0xff is not valid UTF-8
With this, tested
After: 0x41 decodes to 'A'
What Pro unlocks here
- The exact prevention SQL — copy-paste ready
- Raw psql output captured from the Docker lab
- A senior-DBA action list to take it further
- Live monitoring queries to catch it in production
- The deeper audit: fix-that-fails counterexample, GUC before/after, server-log evidence
Related & next steps
Follow the thread
Everything this error touches — jump straight to the sibling error, term, runbook, or parameter.
Verification
- Last verified
- 2026-07-24 (isolated lab, PostgreSQL 18.4)
- Reviewed by
- Verified against PostgreSQL 18.4 in an isolated lab environment
- Audit status
- reviewed