[s3ql] does s3ql work with wasabi?

[Marcin Ciesielski]

Hi

I am using version 2.28 with wasabisys as s3 backend.
few times a day s3ql will report such error (see below)
does it mean that there is a problem with wasabisys compatibility?
------------------------------

ERROR: Uncaught top-level exception:
Traceback (most recent call last):
File "/home/kisiel/s3ql-2.28/src/s3ql/mount.py", line 64, in
run_with_except_hook
run_old(*args, **kw)
File "/usr/lib/python3.5/threading.py", line 862, in run
self._target(*self._args, **self._kwargs)
File "/home/kisiel/s3ql-2.28/src/s3ql/block_cache.py", line 409, in
_upload_loop
self._do_upload(*tmp)
File "/home/kisiel/s3ql-2.28/src/s3ql/block_cache.py", line 436, in
_do_upload
% obj_id).get_obj_size()
File "/home/kisiel/s3ql-2.28/src/s3ql/backends/common.py", line 108, in
wrapped
return method(*a, **kw)
File "/home/kisiel/s3ql-2.28/src/s3ql/backends/common.py", line 340, in
perform_write
return fn(fh)
File "/home/kisiel/s3ql-2.28/src/s3ql/backends/comprenc.py", line 371, in
exit
self.close()
File "/home/kisiel/s3ql-2.28/src/s3ql/backends/comprenc.py", line 365, in
close
self.fh.close()
File "/home/kisiel/s3ql-2.28/src/s3ql/backends/comprenc.py", line 530, in
close
self.fh.close()
File "/home/kisiel/s3ql-2.28/src/s3ql/backends/common.py", line 108, in
wrapped
return method(*a, **kw)
File "/home/kisiel/s3ql-2.28/src/s3ql/backends/s3c.py", line 948, in close
headers=self.headers, body=self.fh)
File "/home/kisiel/s3ql-2.28/src/s3ql/backends/s3c.py", line 550, in
_do_request
self._parse_error_response(resp)
File "/home/kisiel/s3ql-2.28/src/s3ql/backends/s3c.py", line 572, in
_parse_error_response
raise HTTPError(resp.status, resp.reason, resp.headers)
s3ql.backends.s3c.HTTPError: 400 Bad Request: Network error.

[Nikolaus Rath]

Post by Marcin Ciesielski
Hi
I am using version 2.28 with wasabisys as s3 backend.
few times a day s3ql will report such error (see below)
does it mean that there is a problem with wasabisys compatibility?
------------------------------

[...]

Post by Marcin Ciesielski
File "/home/kisiel/s3ql-2.28/src/s3ql/backends/s3c.py", line 572, in
_parse_error_response
raise HTTPError(resp.status, resp.reason, resp.headers)
s3ql.backends.s3c.HTTPError: 400 Bad Request: Network error.

A 400 error means that the client (in this case S3QL) sent a malformed
request. However, if this happens only once in a while then this is very
unlikely. Also, the message "Network Error" doesn't seem to indicate a
client problem.

More likely, there is some problem with Wasabisys or a proxy between you
and them. In either case, it's a bug in the program that sends the error
code. If there is a a temporary problem the server should report
something like 408 (cf https://en.wikipedia.org/wiki/List_of_HTTP_status_codes).


Best,
-Nikolaus

[Marcin Ciesielski]

It happens every few hours a day. I guess maybe every 3-6 hours. I am
transferring a lot of data so it might be the factor.
I was not able to connect this to any specific activity.

After that, I get the transport disconnected message and have to reset the
system.

I don't use any proxy so this cannot be the reason, hence my question if
s3ql does not work with wasabi.
I assume this is not tested. Please correct me if I am wrong.

Would there be a way for me to provide you more detailed logs to see if
that's not error handling on the s3ql side?

Post by Nikolaus Rath

Post by Marcin Ciesielski
Hi
I am using version 2.28 with wasabisys as s3 backend.
few times a day s3ql will report such error (see below)
does it mean that there is a problem with wasabisys compatibility?
------------------------------

[...]

Post by Marcin Ciesielski
File "/home/kisiel/s3ql-2.28/src/s3ql/backends/s3c.py", line 572, in
_parse_error_response
raise HTTPError(resp.status, resp.reason, resp.headers)
s3ql.backends.s3c.HTTPError: 400 Bad Request: Network error.

A 400 error means that the client (in this case S3QL) sent a malformed
request. However, if this happens only once in a while then this is very
unlikely. Also, the message "Network Error" doesn't seem to indicate a
client problem.
More likely, there is some problem with Wasabisys or a proxy between you
and them. In either case, it's a bug in the program that sends the error
code. If there is a a temporary problem the server should report
something like 408 (cf
https://en.wikipedia.org/wiki/List_of_HTTP_status_codes).
Best,
-Nikolaus
--
GPG Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F
»Time flies like an arrow, fruit flies like a Banana.«

[Nikolaus Rath]

Hi Marcin,

A: Because it confuses the reader.
Q: Why?
A: No.
Q: Should I write my response above the quoted reply?

Post by Marcin Ciesielski

Post by Nikolaus Rath

Post by Marcin Ciesielski
Hi
I am using version 2.28 with wasabisys as s3 backend.
few times a day s3ql will report such error (see below)
does it mean that there is a problem with wasabisys compatibility?
------------------------------

[...]

Post by Marcin Ciesielski
File "/home/kisiel/s3ql-2.28/src/s3ql/backends/s3c.py", line 572, in
_parse_error_response
raise HTTPError(resp.status, resp.reason, resp.headers)
s3ql.backends.s3c.HTTPError: 400 Bad Request: Network error.

A 400 error means that the client (in this case S3QL) sent a malformed
request. However, if this happens only once in a while then this is very
unlikely. Also, the message "Network Error" doesn't seem to indicate a
client problem.
More likely, there is some problem with Wasabisys or a proxy between you
and them. In either case, it's a bug in the program that sends the error
code. If there is a a temporary problem the server should report
something like 408 (cf
https://en.wikipedia.org/wiki/List_of_HTTP_status_codes).

It happens every few hours a day. I guess maybe every 3-6 hours. I am
transferring a lot of data so it might be the factor.
I was not able to connect this to any specific activity.
After that, I get the transport disconnected message and have to reset the
system.
I don't use any proxy so this cannot be the reason, hence my question if
s3ql does not work with wasabi.

Just because you're not aware of it doesn't mean you don't use one. Your
ISP may silently pass your requests through its proxy, or Wasabi itself
may be using one or more reverse proxies without you being able to do
anything about it :-).

Post by Marcin Ciesielski
I assume this is not tested. Please correct me if I am wrong.

I have never heard of Wasabi nor did I test S3QL with it. I don't know
if anyone else has. That's all I can say.

Post by Marcin Ciesielski
Would there be a way for me to provide you more detailed logs to see if
that's not error handling on the s3ql side?

You could capture the full server response with a Wireshark to see if
there's more useful information in the body. But most likely you'll have
to contact Wasabi support.

Best,
-Nikolaus