link管理
链接快照平台
- 输入网页链接，自动生成快照
- 标签化管理网页链接
相关文章推荐
|
乐观的卤蛋 · 华为手环6：全天血氧监测，96种运动模式_原 ...· 1 月前 · |
|
|
谦和的移动电源 · 瑞鸣唱片《刘雨婷长笛公主与动画DSD》202 ...· 2 月前 · |
|
|
阳刚的烤地瓜 · 2024女生13款【短髮造型】盤點！一刀切短 ...· 5 月前 · |
|
|
坚强的机器猫 · 从XMind与MindMaster的交互比较 ...· 7 月前 · |
使用Android时，blob的最大大小是什么?这是在哪里记录的？
EN
问
使用Android时，blob的最大大小是什么?这是在哪里记录的？
Stack Overflow用户
提问于
2022-08-28 17:42:40
回答 1
查看 258
关注 0
票数 1
我使用Android并将ByteArray数据存储为BLOB。在一定大小以上，试图检索一行的尝试将失败，其中包含以下消息：
E/CursorWindow: The size of (0, 2) is too big (4194304), so replace the value to NULL坠机的实际原因是：
java.lang.NullPointerException: Parameter specified as non-null is null: method kotlin.jvm.internal.Intrinsics.checkNotNullParameter, parameter bytes因此，在一定大小以上，CursorWindow无法加载blob，返回null，DAO方法抛出NPE，因为实体字段不可空。
我可以理解，在Sqlite/Room中会有一些关于blob大小的上限。在实验上，我确定我的特定设置(至少在某些情况下)是4194304 (4096*1024)。
我的问题是:这些文件在哪里？是否可以根据设备、房间/Sqlite版本等进行更改。我可以在运行时确定这个值吗？
我想在数据插入之前强制执行适当的字节数组限制，这样这个错误就不会发生，我正在寻找一种原则/可靠的方法来确定实际的限制。
更新
我认为@MikeT的答案是我们所能做的最好的。我们可以使用android.database.sqlite检索的最大行大小基本上是没有文档说明的；大多数将其设置为~2MB，但由于某种原因，我(有时)使用Android12将其钉住在~4MB。下面是@MikeT在Kotlin中的答案的一个版本，无论您的安装程序是对超大行抛出异常还是(像我的一样)对列返回一个空值(当使用包含非空字段的Room实体时，它将导致通过道方法抛出的NPE )。
fun checkCursorWindowSize() {
val context = this
val TEST_TABLE_NAME = "tcw"
val TEST_DATABASE_NAME = "tcw"
val TEST_COLUMN_NAME = "testblob"
val reducer = 1024 * 1024 / 16 /* size to reduce BLOB by per iteration */
val allowance =
1 /* size to reduce BLOB by for all extractions (reason for use probably best to not be on a power of 2 boundary????) */
val ba = ByteArray(1024 * 1024 * 4) /* initial size of BLOB when storing*/
var csr: Cursor
val tcwDbPath: String = context.getDatabasePath(TEST_DATABASE_NAME).getPath()
val dbFile: File = context.getDatabasePath(TEST_DATABASE_NAME)
val dbDirectory: File = dbFile.getParentFile()
/* If the database exists then delete it = should never exist */if (dbFile.exists()) {
dbFile.delete()
/* Ensure that the databases directory exists - otherwise openOrCreate fails */if (!dbDirectory.exists()) {
Log.d(
"DEBUGINFO",
"Database directory " + dbDirectory.getPath()
.toString() + " does not exist - Creating"
dbDirectory.mkdirs()
/* Finally create the database */
val db = SQLiteDatabase.openOrCreateDatabase(tcwDbPath, null, null)
/* Do everything inside a transaction to reduce writes */db.beginTransaction()
/* Create the table into which the BLOB will be loaded */db.execSQL("CREATE TABLE IF NOT EXISTS $TEST_TABLE_NAME ($TEST_COLUMN_NAME BLOB)")
/* Insert the over-sized BLOB */
val cv = ContentValues()
cv.put(TEST_COLUMN_NAME, ba)
db.insert(TEST_TABLE_NAME, null, cv)
/* Prepare to repeatedly extract an eve decreasing part of the blob */
var retrieveSize = ba.size /* first try to retrieve entire BLOB */
var caught = 0 /* the number of caught exceptions */
/* Try 1024 attempts at getting smaller BLOB */
for (i in 0..1024) {
Log.d("DEBUG", "Iteration is " + (i + 1) + ". Retrieve Size is " + retrieveSize)
try {
/* Build the query to extract part of the blob (using the SQLite substr function) according to
the current retrieve size less the allowance
only getting 1 row (should only be 1 row)
NOTE!! retrieving an oversize BLOB does not fail here, rather the failure is when attempting to read the data (getBlob method)
csr = db.query(
TEST_TABLE_NAME,
arrayOf("substr(" + TEST_COLUMN_NAME + ",1," + (retrieveSize - allowance) + ") AS " + TEST_COLUMN_NAME),
null,
null,
null,
null,
null,
var test: ByteArray?=null
while (csr.moveToNext()) {
test = csr.getBlob(0) /* get the Blob from the 1st (only) column */
Log.d("DEBUG", "Received value: ${test?.size}")
if (test != null) {
break
} catch (e: SQLException) {
caught++
//e.printStackTrace();
} finally {
retrieveSize =
retrieveSize - reducer /* reduce the retrieveSize ready for the next iteration (if one )*/
if (retrieveSize < 0) break
Log.d("DEBUGINFO", "Caught $caught SQlite Exceptions")
/* Done with the database so clean up */db.endTransaction()
db.close()
File(tcwDbPath).delete()
/* adjust the retrieve size as it has been adjusted in preparation for the next iteration */
Log.d("DEBUGINFO","MaxCursorSize = " + (retrieveSize + reducer).toLong())
}
EN
回答 1
Stack Overflow用户
回答已采纳
发布于 2022-08-28 19:33:38
您的问题不是BLOB的最大大小，而是游标窗口的最大大小，它是特定于Android API/实现的，限制范围从1Mb到4Mb不等(如果我没记错的话)，这取决于安卓的版本(后面的版本具有更大的大小)。
游标窗口包装在游标实现中，对于Room，游标实现封装在方便的方法中，因此确定大小(虽然可能是可能的)可能是一项困难/复杂的任务。
可以绕过这个问题，即游标窗口必须能够存储整行(因此其他列会影响Blob本身的大小)。您可以提取块的部分(块)，然后将它们组合起来。使用小于1Mb的块(可能要少得多)将覆盖所有的限制。
然而，对于如此大量的数据，建议将数据存储为文件，然后存储足够的文件路径，以便能够检索该文件。
这里有两个答案(但不使用Room )，它们从大于游标窗口限制的Blob中检索数据。
- How to use images in Android SQLite that are larger than the limitations of a CursorWindow?
- SQLiteBlobTooBigException still occurs after dividing the request in chunks of 1 MB (and cursor.close())
，你知道我如何确定一个特定版本的Android的限制大小(或者在运行时确定我的应用程序所使用的版本)吗？我在用于Android12的AOSP代码中发现了config_cursorWindowSize = 2048 (这是我目前正在测试的)，但这不符合这样一个事实，即我已经有了高达4MB/ blob的查询(实际上，每一行都是这样)。
从有限的测试来看，(API 28和31)是一种方式，但我怀疑这可能会改变(两者都显示2MB，即2097152)，这取决于其他用途，因为CursorWindow可以使用传递windowSize的构造函数来构造。
下面是基本/有限的java方法
exception
- finally
- 创建一个数据库和一个表，其中只有1列将一个超大的BLOB (4 4Mb)作为唯一行插入，然后
- 将逐步尝试提取BLOB的一个较小的部分(最初都是这样)，直到可以在不使用
- 的情况下提取数据，从而清除数据库
。
:-
long checkCursorWindowSize(Context context) {
final String TEST_TABLE_NAME = "tcw";
final String TEST_DATABASE_NAME = "tcw";
final String TEST_COLUMN_NAME = "testblob";
final int reducer = 1024 * 1024 / 16; /* size to reduce BLOB by per iteration */
final int allowance = 1; /* size to reduce BLOB by for all extractions (reason for use probably best to not be on a power of 2 boundary????) */
byte[] ba = new byte[1024 * 1024 * 4]; /* initial size of BLOB when storing*/
Cursor csr;
String tcwDbPath = context.getDatabasePath(TEST_DATABASE_NAME).getPath();
File dbFile = context.getDatabasePath(TEST_DATABASE_NAME);
File dbDirectory = dbFile.getParentFile();
/* If the database exists then delete it = should never exist */
if (dbFile.exists()) {
dbFile.delete();
/* Ensure that the databases directory exists - otherwise openOrCreate fails */
if (!dbDirectory.exists()) {
Log.d("DEBUGINFO", "Database directory " + dbDirectory.getPath() + " does not exist - Creating");
dbDirectory.mkdirs();
/* Finally create the database */
SQLiteDatabase db = SQLiteDatabase.openOrCreateDatabase(tcwDbPath,null,null);
/* Do everything inside a transaction to reduce writes */
db.beginTransaction();
/* Create the table into which the BLOB will be loaded */
db.execSQL("CREATE TABLE IF NOT EXISTS " + TEST_TABLE_NAME + " (" + TEST_COLUMN_NAME + " BLOB)");
/* Insert the over-sized BLOB */
ContentValues cv = new ContentValues();
cv.put(TEST_COLUMN_NAME,ba);
db.insert(TEST_TABLE_NAME,null,cv);
/* Prepare to repeatedly extract an eve decreasing part of the blob */
int retrieveSize = ba.length; /* first try to retrieve entire BLOB */
int caught = 0; /* the number of caught exceptions */
/* Try 1024 attempts at getting smaller BLOB */
for (int i=0; i <= 1024; i++ ) {
Log.d("DEBUG","Iteration is " + (i + 1) + ". Retrieve Size is " + retrieveSize);
try {
/* Build the query to extract part of the blob (using the SQLite substr function) according to
the current retrieve size less the allowance
only getting 1 row (should only be 1 row)
NOTE!! retrieving an oversize BLOB does not fail here, rather the failure is when attempting to read the data (getBlob method)
csr = db.query(TEST_TABLE_NAME, new String[]{"substr(" + TEST_COLUMN_NAME + ",1," + (retrieveSize - allowance) + ") AS " + TEST_COLUMN_NAME},null,null,null,null,null,"1");
while (csr.moveToNext()) {
byte[] test = csr.getBlob(0); /* get the Blob from the 1st (only) column */
break; /* if no Exception then all done */
} catch (SQLException e) {
retrieveSize = retrieveSize - reducer; /* reduce the retrieveSize ready for the next iteration (if one )*/
caught++;
//e.printStackTrace();
Log.d("DEBUGINFO","Caught " + caught + " SQlite Exceptions");
/* Done with the database so clean up */
db.endTransaction();
db.close();
new File(tcwDbPath).delete();
/* adjust the retrieve size as it has been adjusted in preparation for the next iteration */
return retrieveSize + reducer;
}有限测试在MainActivity中使用了以下内容：
@Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.activity_main);
Log.d("DEBUGINFO","MaxCursorSize = " + checkCursorWindowSize(this));
}
在API 28 :-
D/DEBUGINFO: MaxCursorSize = 2097152
上，整个日志是：-
2022-08-29 10:58:06.798 D/DEBUG: Iteration is 16. Retrieve Size is 3211264
2022-08-29 10:58:06.802 W/CursorWindow: Window is full: requested allocation 3211263 bytes, free space 2096720 bytes, window size 2097152 bytes
2022-08-29 10:58:06.802 E/SQLiteQuery: exception: Row too big to fit into CursorWindow requiredPos=0, totalRows=1; query: SELECT substr(testblob,1,3211263) AS testblob FROM tcw LIMIT 1
2022-08-29 10:58:06.802 D/DEBUG: Iteration is 17. Retrieve Size is 3145728
2022-08-29 10:58:06.806 W/CursorWindow: Window is full: requested allocation 3145727 bytes, free space 2096720 bytes, window size 2097152 bytes
2022-08-29 10:58:06.807 E/SQLiteQuery: exception: Row too big to fit into CursorWindow requiredPos=0, totalRows=1; query: SELECT substr(testblob,1,3145727) AS testblob FROM tcw LIMIT 1
2022-08-29 10:58:06.807 D/DEBUG: Iteration is 18. Retrieve Size is 3080192
2022-08-29 10:58:06.810 W/CursorWindow: Window is full: requested allocation 3080191 bytes, free space 2096720 bytes, window size 2097152 bytes
2022-08-29 10:58:06.810 E/SQLiteQuery: exception: Row too big to fit into CursorWindow requiredPos=0, totalRows=1; query: SELECT substr(testblob,1,3080191) AS testblob FROM tcw LIMIT 1
2022-08-29 10:58:06.810 D/DEBUG: Iteration is 19. Retrieve Size is 3014656
2022-08-29 10:58:06.813 W/CursorWindow: Window is full: requested allocation 3014655 bytes, free space 2096720 bytes, window size 2097152 bytes
2022-08-29 10:58:06.813 E/SQLiteQuery: exception: Row too big to fit into CursorWindow requiredPos=0, totalRows=1; query: SELECT substr(testblob,1,3014655) AS testblob FROM tcw LIMIT 1
2022-08-29 10:58:06.813 D/DEBUG: Iteration is 20. Retrieve Size is 2949120
2022-08-29 10:58:06.818 W/CursorWindow: Window is full: requested allocation 2949119 bytes, free space 2096720 bytes, window size 2097152 bytes
2022-08-29 10:58:06.819 E/SQLiteQuery: exception: Row too big to fit into CursorWindow requiredPos=0, totalRows=1; query: SELECT substr(testblob,1,2949119) AS testblob FROM tcw LIMIT 1
2022-08-29 10:58:06.819 D/DEBUG: Iteration is 21. Retrieve Size is 2883584
2022-08-29 10:58:06.824 W/CursorWindow: Window is full: requested allocation 2883583 bytes, free space 2096720 bytes, window size 2097152 bytes
2022-08-29 10:58:06.824 E/SQLiteQuery: exception: Row too big to fit into CursorWindow requiredPos=0, totalRows=1; query: SELECT substr(testblob,1,2883583) AS testblob FROM tcw LIMIT 1
2022-08-29 10:58:06.825 D/DEBUG: Iteration is 22. Retrieve Size is 2818048
2022-08-29 10:58:06.829 W/CursorWindow: Window is full: requested allocation 2818047 bytes, free space 2096720 bytes, window size 2097152 bytes
2022-08-29 10:58:06.829 E/SQLiteQuery: exception: Row too big to fit into CursorWindow requiredPos=0, totalRows=1; query: SELECT substr(testblob,1,2818047) AS testblob FROM tcw LIMIT 1
2022-08-29 10:58:06.829 D/DEBUG: Iteration is 23. Retrieve Size is 2752512
2022-08-29 10:58:06.832 W/CursorWindow: Window is full: requested allocation 2752511 bytes, free space 2096720 bytes, window size 2097152 bytes
2022-08-29 10:58:06.832 E/SQLiteQuery: exception: Row too big to fit into CursorWindow requiredPos=0, totalRows=1; query: SELECT substr(testblob,1,2752511) AS testblob FROM tcw LIMIT 1
2022-08-29 10:58:06.832 D/DEBUG: Iteration is 24. Retrieve Size is 2686976
2022-08-29 10:58:06.837 W/CursorWindow: Window is full: requested allocation 2686975 bytes, free space 2096720 bytes, window size 2097152 bytes
2022-08-29 10:58:06.838 E/SQLiteQuery: exception: Row too big to fit into CursorWindow requiredPos=0, totalRows=1; query: SELECT substr(testblob,1,2686975) AS testblob FROM tcw LIMIT 1
2022-08-29 10:58:06.838 D/DEBUG: Iteration is 25. Retrieve Size is 2621440
2022-08-29 10:58:06.842 W/CursorWindow: Window is full: requested allocation 2621439 bytes, free space 2096720 bytes, window size 2097152 bytes
2022-08-29 10:58:06.842 E/SQLiteQuery: exception: Row too big to fit into CursorWindow requiredPos=0, totalRows=1; query: SELECT substr(testblob,1,2621439) AS testblob FROM tcw LIMIT 1
2022-08-29 10:58:06.842 D/DEBUG: Iteration is 26. Retrieve Size is 2555904
2022-08-29 10:58:06.846 W/CursorWindow: Window is full: requested allocation 2555903 bytes, free space 2096720 bytes, window size 2097152 bytes
2022-08-29 10:58:06.846 E/SQLiteQuery: exception: Row too big to fit into CursorWindow requiredPos=0, totalRows=1; query: SELECT substr(testblob,1,2555903) AS testblob FROM tcw LIMIT 1
2022-08-29 10:58:06.846 D/DEBUG: Iteration is 27. Retrieve Size is 2490368
2022-08-29 10:58:06.849 W/CursorWindow: Window is full: requested allocation 2490367 bytes, free space 2096720 bytes, window size 2097152 bytes
2022-08-29 10:58:06.849 E/SQLiteQuery: exception: Row too big to fit into CursorWindow requiredPos=0, totalRows=1; query: SELECT substr(testblob,1,2490367) AS testblob FROM tcw LIMIT 1
2022-08-29 10:58:06.849 D/DEBUG: Iteration is 28. Retrieve Size is 2424832
2022-08-29 10:58:06.853 W/CursorWindow: Window is full: requested allocation 2424831 bytes, free space 2096720 bytes, window size 2097152 bytes
2022-08-29 10:58:06.853 E/SQLiteQuery: exception: Row too big to fit into CursorWindow requiredPos=0, totalRows=1; query: SELECT substr(testblob,1,2424831) AS testblob FROM tcw LIMIT 1
2022-08-29 10:58:06.854 D/DEBUG: Iteration is 29. Retrieve Size is 2359296
2022-08-29 10:58:06.857 W/CursorWindow: Window is full: requested allocation 2359295 bytes, free space 2096720 bytes, window size 2097152 bytes
2022-08-29 10:58:06.858 E/SQLiteQuery: exception: Row too big to fit into CursorWindow requiredPos=0, totalRows=1; query: SELECT substr(testblob,1,2359295) AS testblob FROM tcw LIMIT 1
2022-08-29 10:58:06.858 D/DEBUG: Iteration is 30. Retrieve Size is 2293760
2022-08-29 10:58:06.862 W/CursorWindow: Window is full: requested allocation 2293759 bytes, free space 2096720 bytes, window size 2097152 bytes
2022-08-29 10:58:06.862 E/SQLiteQuery: exception: Row too big to fit into CursorWindow requiredPos=0, totalRows=1; query: SELECT substr(testblob,1,2293759) AS testblob FROM tcw LIMIT 1
2022-08-29 10:58:06.862 D/DEBUG: Iteration is 31. Retrieve Size is 2228224
2022-08-29 10:58:06.865 W/CursorWindow: Window is full: requested allocation 2228223 bytes, free space 2096720 bytes, window size 2097152 bytes
2022-08-29 10:58:06.865 E/SQLiteQuery: exception: Row too big to fit into CursorWindow requiredPos=0, totalRows=1; query: SELECT substr(testblob,1,2228223) AS testblob FROM tcw LIMIT 1
2022-08-29 10:58:06.865 D/DEBUG: Iteration is 32. Retrieve Size is 2162688
2022-08-29 10:58:06.870 W/CursorWindow: Window is full: requested allocation 2162687 bytes, free space 2096720 bytes, window size 2097152 bytes
2022-08-29 10:58:06.870 E/SQLiteQuery: exception: Row too big to fit into CursorWindow requiredPos=0, totalRows=1; query: SELECT substr(testblob,1,2162687) AS testblob FROM tcw LIMIT 1
推荐文章