fgets flawed beyond 2GB in BCB Tokyo 10.2 Version 25.0.26309.314
C++ fgets works ok up to offsets of 2GB, but beyond that it returns wrong results (the line read in is not the one I can see is there).
Any remedy? Thanks!
Hi Emanuel, I think you are up against a fundamental limitation of the C libaray here. fgets() will use the typedef fpos_t which is a [signed] long. This will be 32 bits hence max it can get to is your 2GB offset. You don't say which compiler you are using or which target you are compiling for. You don't say why you are using the very old 'C' stream function rather than a C++ or VCL or FireMonkey interface. If you are targeting Windows and can move to Windows 64bit target using the Windows 64bit compiler all these problems should disappear. Regards, Roger.
We, too, are big fans of BCC.
But I think you are on an uphill struggle trying to get it to work beyond 2GB because everything in the libraries is going to be 32 bit integer based. I don't think I'd go there...... You may be braver than me.
Rats, I'm a bloody fool: you're absolutely right, there are multiple compilers in Tokyo - apologies for the confusion!
So, to clarify:
- I'm using the "good'ol" Borland 32-bit BCC. The reason for that is that as far as I tried, the clang debugger is a total catastrophe, it's basically unusable
- I'm using FILE*s for historical reasons. Since about three decades now I'm thoroughly exploiting the beauty of reusability of C++ and as a result, many parts of my latest algorithms are actually driven by high-level libraries I wrote decades ago (and at that time, I had other legacy code written in C, so no streams allowed).
I could perhaps switch from the dated FILE*s to streams, but that would entail rewriting hundreds of thousands of lines - I would obviously prefer to avoid that at any cost.
But your suggestion to use streams instead of FILE*s is quite interesting, given that StackOverflow seems to suggest ways of associating a stream with an existing FILE*. I'll have a look at it.
Thank you for all your help, I really appreciate it! Have a great evening,
Says that for the clang Win64 compiler a long is still 32 bits. (long long is 64 bits).
Because fgets() uses fpos_t for the file pointer and fpos_t is defined as long this is the cause of your problem.
What is the solution?
Well my first thought is to just use the C++ >> streaming operator to read in a line. Is there a reason why you can't do this?
Rem: in case the embarcadero community editor interprets my text as html I remind you that the streaming operator for reading from a stream is two consecutive "greater than" symbols (ie two consecutive closing triangle brackets)
Rem RAD Studio comes with lots of compilers, BCC, clang (win 32), clang (win64) arm (android).... the only clue as to which on you were using is in the tag. Too subtle for me.
So you are using Win 64 clang targeting win64 ? even stranger to choose a C library function for stream input. But I would expect this to work with Win64., surely fpos_t is a long which would be 64 bit for clang 64. I need to check both aspects of this, a) what is fpos_t defined as (delving into the library source code) and b) what is a long on the clang 64 bit compiler. It's late here in the UK (22:42) I may get a chance to do this tomorrow.
Thank you for your reply. Huh, I DID specify the compiler in the very question (i.e. BCB Tokyo 10.2 Version 25.0.26309.314) and thought setting the "win64" tag would make it clear that I was on Win64?
Anyway, I obviously suspected the 2GB apparent limit would be due to a 32 bit long somewhere... BUT, other functions for streams DO have their 64-bit versions, namely _fseeki64() and _ftelli64(), so I wondered whether there was a 64-bit variant of fgets() as well?
All the best, Emanuel