Skill Sprint: Practical Data Compression and Decompression
Delphi Skill Sprint Replay
Background on Data Compression
Most media compression (images, videos, audio, etc.) uses lossy compression, which is specialized compression that removes part of the media in a way that only degrades the media slightly. For data compression, a lossless compression is used so that the data is preserved through the compression process.
Most lossless data compression is based on two papers by Abraham Lempel and Jacob Ziv published in 1977 and 1978. These papers are referred to as LZ77 & LZ78. Derived compression algorithms are often referred to as LZW, LZSS, LZMA, etc.
The .ZIP file format was developed by Phil Katz to store multiple files with different compression options. The most popular compression option was deflate. Lempel–Ziv–Welch (LZW) is the algorithm behind deflate. The Unix compress utility and the GIF image file format were based on same deflate implementation. Later it was discovered this implementation was encumbered by patents.
Gzip was developed as an open and alternative implementation of deflate that avoids the patents on compress and GIF. It is used as the basis for the new PNG image file format.
ZLib is a library that wraps gzip's deflate. It supports 3 modes: raw deflate without headers; Gzip with headers and PNG image file format.
For more information:
- Mark Adler's Stack Overflow post
- Wikipedia on Data compression
- Wikipedia's comparison of archive formats
HTTP Compression Options
- On the request, Accept-Encoding specifies expected compression(s).
- In the response, Content-Encoding specifies the actual compression.
- Most common encodings:
- gzip & deflate (gzip is much more common)
- Other encodings (rare):
- compres, exi, identity, pack200-gzip, xz, bzip2, lzma, xpress
- brotli is a new encoding by Google that is 20% more effective for HTML
- Accept-Encoding on the request may be ignored.
- Stack Overflow always compresses with gzip
Compression Library Options
- Preinstalled
- System.Zlib – supports gzip (with proper header) and deflate
- TCompressionStream, ZCompressStream, ZCompressStr and ZCompress
- Delphi examples
- C++Builder examples
- Indy’s TIdCompressorZLib (also based on Zlib)
-
- HTTP GZip, HTTP Deflate, FTPToIO and FTPDeflate
- System.Zlib – supports gzip (with proper header) and deflate
- Available via GetIt
- Abbrevia (all platforms)
- Huge variety of compressions, including Zip files
- Has nice test suite with usage examples
- Local folder: $(BDSCatalogRepository)\Abbrevia-10.0\tests
-
- BDSCatalogRepository = C:\Users\[user]\Documents\Embarcadero\Studio\17.0\CatalogRepository\
- JEDI JCL – JclCompression (Windows only)
- More compressions, including 7zip files
- Abbrevia (all platforms)
Code Samples


Comments
-
CESAR G. XMAPAS Tuesday, 9 May 2017
Hello. This is what i've been looking for so long, but here's the question:
I have a webbroker server with the apis for my client, everything go ok, but i'm trying to compress my jsonarray data from the server to be used on my client (using the same components as in your example) but i can't get it work fine. So i leave my code from the server side for your help please:
procedure TWModMain.LoginGet(Request: TWebRequest; Response: TWebResponse);
var
a: TJSONArray; //array de json
o: TJSONObject; //objects del json array
MemStr, gZIPStream: TMemoryStream; // for gzip
begin
//Response.Content := ' {Test OK} ' ;
// Load query parameters.
qry.SQL.clear;
if (Request.QueryFields.values['_IdUser'] '') then
begin
qry.SQL.Text := 'SELECT a.* FROM usuarios a ' +
' WHERE a.id_usuario = "' + Request.QueryFields.values['_IdUser'] + '"';
end
else qry.SQL.Text := 'SELECT a.* FROM usuarios a ';
// Put results in a JSON object...
FDConnection1.Connected:=True;
qry.Active := true;
if qry.Active then
begin
if qry.RecordCount>0 then
begin
a := TJSONArray.Create;
try
qry.First;
while (not qry.Eof) do
begin
o := TJSONObject.Create;
o.AddPair('_IdUser',TJSONString.Create( qry.FieldByName('id_usuario').AsString ));
o.AddPair('_Password',qry.FieldByName('Password').AsString );
o.AddPair('_Name',qry.FieldByName('FirstName').AsString + ' ' + qry.FieldByName('LastName').AsString );
a.AddElement(o);
qry.Next;
end;
finally
//****** parameter from the request client side
// compress the data and return gzip file
if Request.QueryFields.values['_zip'] = '' then
begin
//Init
MemStr:= TMemoryStream.Create;
gZIPStream:=TMemoryStream.Create;
try
MemStr.Seek(0, soFromBeginning);
//convert jsonarray to Tstream
WriteStreamStr(MemStr, Utf8ToAnsi(a.ToString));
//make it gzip from (http://stackoverflow.com/questions/40047200/delphi-how-to-set-twebresponse-to-contain-gzip-and-read-it-in-android-app)
doGZIP(MemStr,gZIPStream);
//prepare responsestream and set content encoding and type
Response.Content := '';
Response.ContentStream:=gZIPStream;
Response.ContentEncoding := 'gzip';
Response.ContentType := 'application/json';
finally
//clear...
MemStr.Free;
gZIPStream.free;
end;
end else
begin
Response.ContentType := 'application/json';
Response.Content := Utf8ToAnsi(a.ToString); //a.ToString; //
end;
//close data and release vars
a.DisposeOf;
qry.Active := False;
FDConnection1.Connected:=False;
end;
end;
end;
end; -
Please login first in order for you to submit comments
- Page :
- 1
The problem is that i can't get data on the client side. (RESTRespLog.contentlength = 0 , RESTRespLog.ContentEncoding='gzip')