Questions tagged [gzip]

0

votes
2

answer
19

Views

Iterating through 2 columns of a structured file

I am trying to parse through a section of a huge dataset. The portion of the dataset that I have is a 3GB gzip file. The file is structured so it has x columns and millions of rows. The columns are separated by commas or some sort of common operator so I can read the file. What I want to do is base...
Kunal Rai
1

votes
2

answer
1.4k

Views

How to upload to compress and upload to s3 on the fly with s3cmd

I just found my box has 5% for HDD hard drive left and I have like almost 250GB of mysql bin file that I want to send to s3. We have moved from mysql to NoSQL and not currently using mysql. However I would love to preserve old data before migration. Problem is I can't just tar the files in a loop be...
black sensei
1

votes
2

answer
38

Views

Using csv.DictWriter to output an in-memory gzipped csv file?

I want to use a DictWriter from Python's csv module to generate a .csv file that's compressed using GZip. I need to do this all in-memory, so utilizing local files is out of the question. However, I'm having trouble dealing with each module's type requirements in Python 3. Assuming that I got the ge...
felipecgonc
0

votes
1

answer
17

Views

Using JSON and gzip together in WCF

I need to create an endpoint on the server side (WCF) that will accept requests from the client that will contain headers: Content-Type : application/json Content-Encoding : gzip Accept-Encoding : gzip Is that possible complete without implementing custom MessageEncodingBindingElement? Something lik...
Igor Sychev
1

votes
1

answer
4.9k

Views

nginx gzip compression not working

I have no idea where to place my gzip compression lines within my http block, shown here. http { default_type application/octet-stream; include /etc/nginx/mime.types; log_format main '$remote_addr - $remote_user [$time_local] "$request" ' '$status $body_bytes_sent "$http_referer" ' '"$http_user...
robinnnnn
1

votes
1

answer
1.3k

Views

Enabling gzip compression in .htaccess file

I am trying to enable Gzip compression as per the Google pagespeed suggestions as to how to get my site to load faster. I have tried the suggested addition to my .htaccess file as found on https://varvy.com/pagespeed/enable-compression.html but it does not work correctly. My current .htaccess file c...
J. Ayo
1

votes
1

answer
3.1k

Views

Accept-Encoding:gzip and Content-Encoding:gzip

What is the difference between the two HTTP headers? Accept-Encoding:gzip Content-Encoding:gzip
user8519380
1

votes
2

answer
407

Views

Tomcat 8.5.29 HTTP/2 is not supporting GZIP compression

I am using Tomcat 8.5.29 and using the respective configuration,I have enabled the HTTP2 support for the site. Below is the configuration in server.xml file. When i tried to compare the page load time for the site which is supporting HTTPS 1.1 and HTTP2, it is not consistent. Sometime it is taking m...
sarita
1

votes
1

answer
33

Views

How to read a GZIP payload in Ruby Sinatra

On a remote host I have a bash script sending a simple gzipped YAML file to my Ruby Sinatra endpoint: #!/bin/bash /bin/gzip -c /tmp/test.yaml > /tmp/test.gz curl -i :/last_run_report -H "Content-Type: application/xml" -H "Content-Encoding: gzip" --data-binary @/tmp/test.gz My sample Ruby app is: req...
haunm
1

votes
1

answer
664

Views

Is it possible to create valid gzip with static dictionary?

I am trying to create valid gzip file (which can be decompressed using standard linux gzip) with data encoded using DEFLATE algorithm with static/preset dictionary. I've read both specifications for DEFLATE and gzip, and it looks like it is impossible. As I get from DEFLATE specification there are t...
dbolotin
1

votes
1

answer
23

Views

Read gz file and get the last 24 hrs line python

I have three file: 2 .gz files and 1 .log file. These files are pretty big. Below I have a sample copy of my original data. I want to extract the entries that correspond to the last 24 hours. a.log.1.gz 2018/03/25-00:08:48.638553 508 7FF4A8F3D704 snononsonfvnosnovoosr 2018/03/25-10:08:48.98...
user15051990
1

votes
3

answer
1.5k

Views

Appending to gzip file in Python doesnt work

I have following function : def save(msg): with gzip.open("ircbot.log.gz", "ab") as f: f.write(msg+'\n') f.close() return "Succesfully logged: "+msg I want to add each msg to .log file but it doesnt work, only first msg is saved. For example after calling those functions : save('first') save('secon...
1

votes
1

answer
1.2k

Views

java - creating empty GZIP file

i am trying to solve a simple problem - creating an empty GZip file using Java, so that no excpetion is raised when trying to read from it. if I do: Files.createFile(outPutFile); new PrintWriter(new GZIPOutputStream(new FileOutputStream(outPutFile.toFile())), true).close(); It solves the problem - i...
Bober02
1

votes
2

answer
1k

Views

Remove .gz Extension

I have a Jekyll site that processes all HTML, CSS and Javascript into *.css.gz, *.js.gz, etc files. The problem is, I need to remove the gz extension so the files can be uploaded to S3 and served using GZip. In my makefile, I have this: find _site/ -iname '*.html' -exec gzip -n {} + find _site/ -ina...
Andrew M
1

votes
1

answer
1.3k

Views

Microsoft.Samples.GZipEncoder with [WebGet(ResponseFormat=WebMessageFormat.Json]?

With the help of this question and this post I've managed to get WebHttpBinding to work with compression (by means of copy-pasting the code). When pointing the browser to my service method, it downloads a file which I can rename to .zip and decompress, so the compression part works. But I'm not able...
Lescai Ionel
1

votes
1

answer
119

Views

Gzipped CSS/JS/HTML

I've heard about this recently, and I have a few questions: Does any browser support this? What happens if a browser doesn't support it, are they left unloaded? Can I use this feature on a shared hosting using Apache and PHP5? How can I use this feature? P.S. It'd be related to Wordpress, if that ma...
Jack
1

votes
3

answer
1.1k

Views

Concatenate gzipped byte arrays in C#

I have gzipped data which is stored in DB. Is there a way to concatenate say 50 separate gzipped data into one gzipped output which can be uncompressed? The result should be same as decompressing that 50 items, concatenating them and then gzipping them. I would like to avoid decompression phase. Is...
Marka
1

votes
2

answer
2.1k

Views

hadoop strategy with big gzip files

I will copy a lot of big weblogs file compressed as gzip in hadoop. I will need run many map/reduce on these file. To my understanding, only one Map/Reduce will be run by file. In my case, it's not acceptable because we need these job to complete as quickly as possible. Is it common practice to spl...
rguillemette
1

votes
1

answer
1.2k

Views

WebSockets and FireFox

Hi everyone and thanks in advance, I'm rolling my own WebSocket server in C++ (I've also quickly rolled one in C# this morning to double check what's going on here) as per RFC 6455 I've read the standard and I now have a server that will successfully form a connection with a client. I can complete t...
Pez
1

votes
1

answer
2.6k

Views

nginx gzip not working on browser but curl works

my environment is nginx(1.3.11) + php-fpm curl -I -H "Accept-Encoding: gzip,deflate" http://www.ihezhu.com/ the result is HTTP/1.1 200 OK Server: nginx Date: Wed, 03 Jul 2013 07:47:27 GMT Content-Type: text/html; charset=utf-8 Connection: keep-alive Vary: Accept-Encoding Set-Cookie: PHP...
wingoo
1

votes
1

answer
265

Views

CPU performance affect GZIP IIS?

I have installed Static and dynamic compression for IIS8 and enabled it. When My CPU performance 100%, My GZIP IIS didn't working, why?? I can clearly see that GZIP is not running in Live HTTP Header. What am I missing here? Does IIS stop performing gzip operation if CPU usage is high? NotGzip
user4951
1

votes
2

answer
551

Views

Do I need use Gzip in a Rails 4 project?

I'm running some audits in my open-source project, BTC-Stores, and some times Chrome shows me that I need "Enable Gzip compression". Sometime ago I read "High Performance Websites", from Steve Souders, and I already know the basic concepts about how make your page load faster. My project is using R...
Paladini
1

votes
1

answer
4.6k

Views

'Not in gzip format" error when decompressing gzip inputstream in java

My code is attempting to decompress an input stream read from a gzipped file. Here is the code snippet: InputStream is = new GZIPInputStream(new ByteArrayInputStream(fcontents.getBytes())); The file itself is fine: $cat storefront3.gz | gunzip 180028796 80026920 180028796 180026921 8002790180 80000...
javadba
1

votes
1

answer
1.7k

Views

ASP.NET, GZip and .AXD dynamic files not playing nicely

I have feebly attempted to enable gzip compression for an (asp.net webforms) web site that I'm working on. I thought that I had everything working, but when I got to dynamic loading of webresource.axd and scriptresource.axd files, they come through compressed and as such the page either fails to lo...
David Pine
1

votes
2

answer
1.6k

Views

GZIP traffic via websockets

Is it possible to pack and unpack websockets traffic frame by frame with GZIP? We have a lot of websockets traff (real time quotes) that should be packed to reduce traffic, but I have no idea if it is even possible.
Prosto Trader
1

votes
1

answer
5.1k

Views

gzip compression not working on xampp

for some reason the gzip is not working my gzip compression script mod_gzip_on Yes mod_gzip_dechunk Yes mod_gzip_item_include file .(html?|txt|css|js|php|pl)$ mod_gzip_item_include handler ^cgi-script$ mod_gzip_item_include mime ^text/.* mod_gzip_item_include mime ^application/x-javascript.* mod_gz...
Immortal Dude
1

votes
1

answer
1k

Views

Cannot gzip compress static files in Nodejs

Gzipping static files does not work as required (as I think). I used gzippo and express.compress(). Both gzipp the files one time. There is no Content-Encoding:gzip if I refresh the page again. Her is how I setup my express app: var gzippo = require('gzippo'); var express = require('express'); var a...
Maxali
1

votes
2

answer
325

Views

HTML template gzip doesn't work (PageSpeed Insights)

I would like to ask how to make compression working on HTML template on 000webhost.com I created .htaccess file and added this: mod_gzip_on Yes mod_gzip_dechunk Yes mod_gzip_item_include file .(html?|txt|css|js|php|pl)$ mod_gzip_item_include handler ^cgi-script$ mod_gzip_item_include mime ^text/.* m...
testerius
1

votes
2

answer
2k

Views

Can I make Fiddler unzip outbound traffic?

In order to replicate a problem some users are having with my application, I need to simulate a corporate proxy server which decompresses GZIP'd http traffic and doesn't recompress it before passing it to the destination server. The users are getting the error "the magic number in the GZIP header i...
NickG
1

votes
2

answer
1.5k

Views

multi cpu core gzip a big file

How can I use all cpu cores in my server(has 4 cores) linux Debian over OpenVZ to gzipping faster one big file? I am trying to use these commands but I can not put the pieces together get number of cores CORES=$(grep -c '^processor' /proc/cpuinfo) this for split big file in more split -b100 file.big...
StefanoCudini
1

votes
2

answer
448

Views

find command - syntax with -exec handle

Background and problem: I am using a find command to gzip all files that match my criteria: sudo -u userAccount -i find /apps/server/userAccount/logs/. -maxdepth 1 -type f -exec gzip {} \; This will go ahead and compress all files against my criteria but I don't want to be prompted to overwrite -- I...
PuffinMaster
1

votes
1

answer
471

Views

Is it possible to make libcurl compressing data of the http messages?

I make a search and I found that the libcurl is able to decompress received data (compressed with gzip for example) using the following option: curl_easy_setopt(curl, CURLOPT_ACCEPT_ENCODING, "gzip"); But I can not find how to make the libcurl sending a compressed data of the http messages? Does th...
MOHAMED
1

votes
2

answer
235

Views

Getting a compressed Neo4j response

I'm using Transactional Cypher HTTP endpoint from my application to execute queries in Neo4j. I was wondering if there is a way to get zipped response from server. I read some threads about it but they mentioned to create unmanaged extensions for it: http://www.markhneedham.com/blog/2013/07/08/neo4j...
Rahul
1

votes
2

answer
827

Views

How can I convert .tar to .tgz

I'm trying to compress some files with python. I'm using subprocess, the problem is that I don't know how to compress all the files. Here is what I do: comand = czf saveFolder = /home/albert/a.tgz temporal = /home/albert/test1 subprocess.call(["tar",comand,saveFolder,temporal]) How can I add more fi...
user2911701
1

votes
1

answer
650

Views

Gzip CSS/JS Using OS X Terminal

I am trying to use OSX Terminal to create gzip versions (.gz) of all css & js files in a folder. I found the following command, but when I cd to a test folder & then enter the command, it doesn't output anything & I would expect it to create a gzip copy in the folder next to the original file: find...
parti
1

votes
2

answer
693

Views

Gulp SourceMaps not working when using gulp-gzip

I am using Gulp to uglify javascript files and generate their source map. So far so good: var gulp = require('gulp'); var plugins = require('gulp-load-plugins')(); gulp.task('compress-js', ['clean-js'], function() { return gulp.src('./resources/js/**/*.js') .pipe(plugins.sourcemaps.init()) .pipe(plu...
codependent
1

votes
1

answer
924

Views

Golang: Why does the compress/gzip Read function not read file contents?

I made a text file, that I then compressed with gzip. I then run the following go program to read the contents of that compressed file. package main import ( "compress/gzip" "fmt" "os" ) func main() { handle, err := os.Open("zipfile.gz") if err != nil { fmt.Println("[ERROR] File Open:", err) } defer...
SunSparc
1

votes
2

answer
403

Views

Improvements in gzip decompression java

Scenario : I have almost close to 15 million records in oracle database, each of that record has a column which is compressed. The task is to export the same table but with the column value decompressed. My solution steps are given below, - Read a chunk of data using jdbcTemplate (returns List) - Fo...
Kiran Joshi
1

votes
1

answer
879

Views

Bottle/Python - “IOError: [Errno 9] read() on write-only GzipFile object” when trying to decompress request body

I use Bottle to receive gzipped request body. When the size of the request is small, everything is fine. However, if the size of request body is a bit larger (say, >= 20kb), an IOError is thrown. The following is the code used to read and decompress the request body: @post("/receive") def receive():...
shihpeng
1

votes
2

answer
29

Views

what is the difference between using “--stdout” and not using it in gzip?

The below is the code that makes me curious about --stdout of the gzip. Using the diff command on the outputs from two different commands tells me the two files are different, but the manual check says otherwise. What's going on? $echo "test" > tmpx $cat tmpx | gzip > tmpx1.gz $cat tmpx | gzip --st...
Alby

View additional questions