invalid byte sequence for encoding "UTF8": 0x00 · Issue #1422 · npgsql/npgsql

link管理

链接快照平台

输入网页链接，自动生成快照
标签化管理网页链接

相关文章推荐

从未表白的围巾 · 【C++快速上手】数值转换字符串方法小结_c ...· 4 天前 ·

机灵的烤地瓜 · 显示器KVM易用模式，共享键鼠，自由切换！_ ...· 4 周前 ·

坏坏的保温杯 · “LAMMPS分子动力学及ReaxFF反应力 ...· 2 月前 ·

精明的台灯 · 哥布林_哔哩哔哩_bilibili· 5 月前 ·

强健的蚂蚁 · Vue3基础(十ba)___在css中使用p ...· 5 月前 ·

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement . We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Hello,

Business Logic:
Read an XML file from an AWS S3 bucket, use Npgsql to execute a "COPY FROM STDIN (FORMAT BINARY)" to do bulk copying into a database/table. Before using NpgsqlBinaryImporter.Write(), I am making sure that I am cleansing data to convert to required UTF-8 format, but I am receiving an exception when the binary importer tries to close() and write the data to my table (exception below).

Note:
This exception occurs randomly and does not always occur when writing data to the database. I've verified that the data it is trying to write is in proper format and if there are any null values, to write as string "NULL" but I can seem to figure out what exactly is causing the issue.

Is there any one else experiencing similar issues with writing using the "COPY" command?

Exception:

Npgsql.PostgresException: 22021: invalid byte sequence for encoding "UTF8": 0x00
   at Npgsql.NpgsqlConnector.DoReadMessage(DataRowLoadingMode dataRowLoadingMode, Boolean isPrependedMessage)
   at Npgsql.NpgsqlConnector.ReadMessageWithPrepended(DataRowLoadingMode dataRowLoadingMode)
   at Npgsql.NpgsqlConnector.ReadExpecting[T]()
   at Npgsql.NpgsqlBinaryImporter.Close()
   at RE.Importer.School.PostgresSQLAdapter.Dispose()
C# .NET Core

Npgsql version: 3.1.7

PostgreSQL version: 9.5.2

Operating system: macOS Sierra 10.12.3
          Before diving into this, could you please try doing the same with the recently-released Npgsql 3.2.0? There were changes that should have removed some protocol sync bugs, just wanting to rule that out.
If not, PostgreSQL seems to be telling you that you sent a 0 character in one of your strings, which isn't allowed. I understand that you're cleansing your data but it's worth making sure. I'd narrow it down to the offensive string - removing rows and rows from the import until you're locked on the row which triggers the error, then the column. Once you have a specific string value triggering this (with the others working), you can see what's wrong