java.util.zip.DataFormatException: incorrect header check #1055
Comments
chengdedeng commented Mar 7, 2019
rocketmq version:4.3.2
error information in the console:
java.lang.NullPointerException
at org.apache.rocketmq.broker.transaction.queue.TransactionalMessageBridge.getMessage(TransactionalMessageBridge.java:136)
at org.apache.rocketmq.broker.transaction.queue.TransactionalMessageBridge.getHalfMessage(TransactionalMessageBridge.java:108)
at org.apache.rocketmq.broker.transaction.queue.TransactionalMessageServiceImpl.pullHalfMsg(TransactionalMessageServiceImpl.java:379)
at org.apache.rocketmq.broker.transaction.queue.TransactionalMessageServiceImpl.getHalfMsg(TransactionalMessageServiceImpl.java:444)
at org.apache.rocketmq.broker.transaction.queue.TransactionalMessageServiceImpl.check(TransactionalMessageServiceImpl.java:164)
at org.apache.rocketmq.broker.transaction.TransactionalMessageCheckService.onWaitEnd(TransactionalMessageCheckService.java:76)
at org.apache.rocketmq.common.ServiceThread.waitForRunning(ServiceThread.java:121)
at org.apache.rocketmq.broker.transaction.TransactionalMessageCheckService.run(TransactionalMessageCheckService.java:65)
at java.lang.Thread.run(Thread.java:748)
I found issue at uncompress of body in the MessageDecoder.java by debug broker code，get error info:
java.util.zip.DataFormatException: incorrect header check
The text was updated successfully, but these errors were encountered:
We are unable to convert the task to an issue at this time. Please try again.
The issue was successfully created but we are unable to update the comment at this time.
[SOLVED] Getting kicked from every Minecraft server with the same error codes.
Agent Dash
Honorable
I keep getting kicked from every server I join every few seconds with different errors every time. The main ones are Badly Compressed Packet, Incorrect Header Check, and Varint too big, and it happens no matter what I do in game and isn’t linked to anything specific. I have made sure to clean up my PC and uninstall any program that may cause issues with MC and I also reinstalled the game.
Last year when playing 1.13.XX Circa November 2018 I never had this issue and not only was I playing nonstop but I since then have not installed anything new besides new steam games. Internet gets about 20 up and 20 down, and I have a PC with a 1080ti, 128 GB ram, and a 6850k processor. I have searched all over google for a fix and have posted to other websites about my problem. No fix found and it’s really getting on my nerves. anyone got any ideas?
Third-Eye
Splendid
I keep getting kicked from every server I join every few seconds with different errors every time. The main ones are Badly Compressed Packet, Incorrect Header Check, and Varint too big, and it happens no matter what I do in game and isn’t linked to anything specific. I have made sure to clean up my PC and uninstall any program that may cause issues with MC and I also reinstalled the game.
Last year when playing 1.13.XX Circa November 2018 I never had this issue and not only was I playing nonstop but I since then have not installed anything new besides new steam games. Internet gets about 20 up and 20 down, and I have a PC with a 1080ti, 128 GB ram, and a 6850k processor. I have searched all over google for a fix and have posted to other websites about my problem. No fix found and it’s really getting on my nerves. anyone got any ideas?
incorrect header check when zlib decompression in http #181
Comments
AdamMagaluk commented Mar 16, 2018
In some cases incorrect header check is thrown when decompressing gzip http(s) data.
We noticed this when using superagent node_module as it sends Accept-Encoding: gzip,deflate by default.
More details to come.
The text was updated successfully, but these errors were encountered:
We are unable to convert the task to an issue at this time. Please try again.
The issue was successfully created but we are unable to update the comment at this time.
AdamMagaluk commented Mar 19, 2018 •
In ZLib.java Trireme converts the Buffer.BufferImpl to a ByteBuffer using chunk.getBuffer() which calls return ByteBuffer.wrap(buf, bufOffset, bufLength); In that process the offset property on ByteBuffer which HeapByteBuffer.java uses with:
Zlib uses the new ByteBuffer to check if the first two bytes equal the magic header gzip uses. see
Because the offset is not set on the ByteBuffer it fails to see the header and errors out.
AdamMagaluk commented Mar 20, 2018 •
The simplest way to illustrate the issue is using the following snippet. This is essentially how Trireme’s modules/Buffer works storing the underlying data in byte[] with a offset/len index since the underlying array may have extra data.
When Triremes public ByteBuffer getBuffer() is called it calls ByteBuffer.wrap with the offset but that does not work properly.
I’m not sure Trireme uses the same concept of offset as ByteBuffer does or ByteBuffer is broken (unlikely). If you look at the constructor to HeapByteBuffer which wrap calls it calls:
But that super is and offset is always set to 0.
incorrect header check #2406
Comments
efreibe commented Sep 12, 2019 •
Hello, why this code is throwing «incorrect header check»?
I tested in the browser (even in incognito mode) and is returning an image.
Changed responseType to ‘blob’ or ‘stream’ but I cannot get rid of that exception.
The text was updated successfully, but these errors were encountered:
We are unable to convert the task to an issue at this time. Please try again.
The issue was successfully created but we are unable to update the comment at this time.
efreibe commented Sep 13, 2019 •
I’ve discovered that the site is sending a header content-encoding: gzip but the data is sent in plain text, so the libray fails to decompress that data.
There is a workaround to consume this response or remove that header before data is gzipped?
I’ve tried creating an interceptor but the code is run after the decompressing code and keeps failing.
doplic commented Sep 25, 2019 •
I’ve run into this same issue. It looks like the code that attempts to process the compressed data is here:
https://github.com/axios/axios/blob/master/lib/adapters/http.js
Line 186-190
The axios version I’m using:
«axios»: «^0.19.0»
I did a quick ghetto fix adding the following error catching on the decompression code to my local axios library. It just passes the stream data onto the next steps if the decompression fails. It’s working for me, but maybe not a committable solution.
efreibe commented Sep 29, 2019
Maybe the solution is to analyze the stream and look for the PK signature.
doplic commented Oct 6, 2019
Axios tries to handle all that for you by checking the server response headers. The decompression is all taken care of by the zlib library via the «createUnzip()» call in Axios (see Zlib createUnZip() ). So the Axios library says to the zlib library «decompress this response data for me because the server headers indicate it’s compressed», and the Zlib library returns «this isn’t compressed data». Axios just passes that error up to the user, and I don’t know of a way of catching that error other than modifying the Axios library code.
Not to steer people away from the Axios library, but I had a much easier time with the ‘request’ library for edge-cases like the one described above: https://www.npmjs.com/package/request
yagodorea commented Feb 6, 2020
Did someone find any solution for this? I’m facing exactly the same issue integrating with MercadoPago.
efreibe commented Feb 7, 2020
Please, use @doplic’s solution. It’s working for MercadoPago.
chinesedfan commented Mar 24, 2020
If someone can learn how package request does, glad to hear that.
У меня есть файл gzip, и я пытаюсь прочитать его через Python, как показано ниже:
он выдает эту ошибку:
Как я могу его преодолеть?
ОТВЕТЫ
Ответ 1
Обновить: dnozay answer объясняет проблему и должен быть принятым ответом.
Ответ 2
У вас есть эта ошибка:
Что наиболее вероятно, потому что вы пытаетесь проверить заголовки, которых там нет, например, ваши данные следуют RFC 1951 (сжатый формат deflate ), а не RFC 1950 (сжатый формат zlib ) или RFC 1952 (сжатый формат gzip ).
выбирая windowBits
Но zlib может распаковать все эти форматы:
Примеры
очевидный тест для zlib :
данные также совместимы с модулем gzip :
автоматическое определение заголовка (zlib или gzip)
добавление 32 к windowBits вызовет обнаружение заголовка
используя вместо этого gzip
Ответ 3
Я просто решил проблему с неправильной проверкой заголовка при распаковке gzipped-данных.
Да, это может быть очень неприятно. Обычно мелкое чтение документации представляет Zlib как сжатие API для Gzip, но по умолчанию (не используя методы gz *) он не создает или не разжимает формат Gzip. Вы должны отправить этот не очень заметный документ.
Ответ 4
Мой случай состоял в том, чтобы распаковать почтовые сообщения, которые хранятся в базе данных Bullhorn. Фрагмент следующий:
Ответ 5
Как ни странно, у меня была эта ошибка при попытке работать с API с помощью Python.
Мне удалось заставить его работать с объектом GzipFile из каталога gzip, примерно так:
Ответ 6
Чтобы распаковать неполные сжатые байты, которые находятся в памяти, полезен ответ dnozay, но он пропускает вызов zlib.decompressobj который я счел необходимым:
Ответ 7
Просто добавьте заголовок «Accept-Encoding»: «identity»