I'm getting an integer overflow error when writing more than about 2^30 characters in a tStringList using
SaveToFile in 64-bit Delphi 10.4.1. What I think is happening is that it is using two bytes per character
and this is putting some number over 2^31 and overflowing an integer.
Is this one of the things that they haven't made 64-bit yet?
If there are under about 1 billion characters, it works. If there are more, it crashes.
Which brings up another of my pet peeves: 64-bit Delphi has never handled integer overflows properly.
A sample program is attached.
SampleProgram.txt
Are you asking about my pet peeve about 64-bit Delphi not handling integer overflows? It has always had this flaw in the 64-bit version. If you have an integer overflow in the EXE, it just stops, with no error message, If you are running it in the IDE, it doesn't take you to the line the error, as it does in the 32-bit version, and, as far as I know, it does with all other runtime errors. An integer overflow in the 64-bit version in the IDE takes you to the obscure place in my message.
When tested using the 64 bit the call stack is:
:00007FF965893E49 ; C:\WINDOWS\System32\KERNELBASE.dll
System._RaiseAtExcept(???,???)
System.SysUtils.ErrorHandler(???,$70AE52)
System.ErrorAt(5,$70AE52)
System._IntOver
mainFRM.TForm1.Button1Click(???)
Vcl.Controls.TControl.Click
Vcl.StdCtrls.TCustomButton.Click
Vcl.StdCtrls.TCustomButton.CNCommand(???)
System.TObject.Dispatch((no value))
2.) The overflow occurs in GetTextStr. When debugging a 32 bit application you get the following call stack:
:766f9ab2 KERNELBASE.RaiseException + 0x62
:41169900
:41169900
System.Classes.TStrings.SaveToStream($3033D38,???)
System.Classes.TStrings.SaveToFile(???,nil)
System.Classes.TStrings.SaveToFile(???)
mainFRM.TForm1.Button2Click($2FCEEA0)
Vcl.Controls.TControl.Click
Vcl.StdCtrls.TCustomButton.Click
SaveToStream is focused, this almost ok as this method calls GetTextStr. I've tried again to run the 64 bit version and I get a different call stack this time:
System._RaiseAtExcept(???,???)
System.SysUtils.ErrorHandler(???,$40FA0A)
System.ErrorAt(5,$40FA0A)
System._IntOver
System._NewUnicodeString(1092000000)
System._UStrFromPWCharLen('',nil {#0},???)
System.Classes.TStrings.GetTextStr
System.Classes.TStrings.SaveToStream($302DD30,$302CDD0)
System.Classes.TStrings.SaveToFile('d:\temp\StringTest.txt',nil)
System.Classes.TStrings.SaveToFile(???)
mainFRM.TForm1.Button2Click(???)
The IDE focuses System._IntOver instead of System._NewUnicodeString.
Other than for certain debugging situations, you should never have a need to turn optimizations off. All optimizations performed by the Delphi compiler are guaranteed not to alter the meaning of a program. In other words, the compiler performs no "unsafe" optimizations that require special awareness by the programmer. This option can only turn optimization on or off for an entire procedure or function. You cannot turn optimization on or off for a single line or group of lines within a routine.
{$APPTYPE CONSOLE}
(* Windows 64-bit platform
It takes about 20 seconds to write the file to an SSD.
number := 20637552 - works
Number := 21000000 - Integer overflow
IDE stops on line in system unit:
procedure _IntOver;
{$IFDEF PUREPASCAL}
begin
ErrorAt(Byte(reIntOverflow), ReturnAddress);
I believe it may be caused in:
system.classes, procedure TStrings.SaveToStream *)
{$R *.res}
// .... R+,Q+,O- // removed by my test
{ R+ }
{ Q+ }
System.SysUtils,
System.classes,
Vcl.Dialogs,
System.Diagnostics;
i, number : integer;
s : string;
sList : tStringList;
FileName : tFileName;
lStartTimer: TStopWatch;
begin
lStartTimer := TStopWatch.StartNew;
FileName := '.\StringTest.txt';
s := '';
for i := 1 to 50 do
s := s + 'A';
number := 20637552; // OK
// number := 21000000; // causes integer overflow, even in 64-bit
sList := tStringList.Create;
for i := 1 to number do
sList.add(s);
writeln('Writing ', length(sList[0]) * number, ' characters');
// at 32bits, the error occurr at:
// System.Classes.pas, line 6815 = SetString(Result, nil, Size); // Out Memory
sList.SaveToFile(FileName); // integer overflow if this line is present
except
on E: Exception do
ShowMessage('error :' + E.ClassName + sLineBreak + E.Message);
finally
if not(sList = nil) then
sList.free;
lStartTimer.Stop;
writeln('Time Elapsed: ' + lStartTimer.ElapsedMilliseconds.ToString);
writeln('File written to ', FileName);
writeln('test done, number = ' + number.ToString);
readln;
When tested using the 64 bit the call stack is:
:00007FF965893E49 ; C:\WINDOWS\System32\KERNELBASE.dll
System._RaiseAtExcept(???,???)
System.SysUtils.ErrorHandler(???,$70AE52)
System.ErrorAt(5,$70AE52)
System._IntOver
mainFRM.TForm1.Button1Click(???)
Vcl.Controls.TControl.Click
Vcl.StdCtrls.TCustomButton.Click
Vcl.StdCtrls.TCustomButton.CNCommand(???)
System.TObject.Dispatch((no value))
Only in 32bits, we have the Out Memory because: "strings" -- NOTE: until NOTEPAD++ crash when try open your file-resulted.txt with more than size 1GB with "AAAAAAAAAAAAAAA....."
I have 16GB RAM and CPU 4x8 i7 4770K and is not enought for open the file, because, the software used try read the file whole on memory... Software like MSWord or similar, do paging to open big-files!
System.Classes.pas, line 6815 = SetString(Result, nil, Size); // Out Memory
as I show in my sample above!
Only in 32bits, we have the Out Memory because: "strings" -- NOTE: until NOTEPAD++ crash when try open your file-resulted.txt with more than size 1GB with "AAAAAAAAAAAAAAA....."
I have 16GB RAM and CPU 4x8 i7 4770K and is not enought for open the file, because, the software used try read the file whole on memory... Software like MSWord or similar, do paging to open big-files!
Only in 32bits, we have the Out Memory because: "strings" -- NOTE: until NOTEPAD++ crash when try open your file-resulted.txt with more than size 1GB with "AAAAAAAAAAAAAAA....."
I have 16GB RAM and CPU 4x8 i7 4770K and is not enought for open the file, because, the software used try read the file whole on memory... Software like MSWord or similar, do paging to open big-files!
number := 20637552; // OK
// number := 21000000; // causes integer overflow, even in 64-bit
AssignFile(myFile, '.\myFileWithStrings.txt');
Rewrite(myFile); // create and open it!
writeln(lPlatform + ', writing string ' + (number * 50).tostring + ' string on ' + '.\myFileWithInteger.txt');
for i := 1 to number do
begin
for z := 1 to 50 do
writeln(myFile, s);
if (i mod 1000000) = 0 then // 1.000.000 each...
writeln('was writed ' + (50 * i).tostring + ' strings');
except
on E: Exception do
ShowMessage('error ...');
finally
CloseFile(myFile);
lStartTimer.Stop;
writeln('Time Elapsed: ' + lStartTimer.ElapsedMilliseconds.tostring);
// writeln('File written to ', FileName);
writeln('test done, number = ' + number.tostring);
readln;
