Problem:
You are trying to use boto3′ upload_fileobj()
to upload a file to S3 storage using code such as
# Create connection to Wasabi / S3 s3 = boto3.resource('s3', endpoint_url = 'https://minio.mydomin.com', aws_access_key_id = 'ACCESS_KEY', aws_secret_access_key = 'SECRET_KEY' ) # Get bucket object my_bucket = s3.Bucket('my-bucket') # Upload string to file with open("testtext.txt", "r") as f: my_bucket.upload_fileobj(f, "test.txt")
But when you try to run it, you see the following stacktrace:
Traceback (most recent call last): File "/home/uli/dev/MyProject/put.py", line 13, in <module> my_bucket.upload_fileobj(f, "test.txt") File "/usr/local/lib/python3.10/dist-packages/boto3/s3/inject.py", line 678, in bucket_upload_fileobj return self.meta.client.upload_fileobj( File "/usr/local/lib/python3.10/dist-packages/boto3/s3/inject.py", line 636, in upload_fileobj return future.result() File "/usr/local/lib/python3.10/dist-packages/s3transfer/futures.py", line 103, in result return self._coordinator.result() File "/usr/local/lib/python3.10/dist-packages/s3transfer/futures.py", line 266, in result raise self._exception File "/usr/local/lib/python3.10/dist-packages/s3transfer/tasks.py", line 139, in __call__ return self._execute_main(kwargs) File "/usr/local/lib/python3.10/dist-packages/s3transfer/tasks.py", line 162, in _execute_main return_value = self._main(**kwargs) File "/usr/local/lib/python3.10/dist-packages/s3transfer/upload.py", line 758, in _main client.put_object(Bucket=bucket, Key=key, Body=body, **extra_args) File "/usr/local/lib/python3.10/dist-packages/botocore/client.py", line 530, in _api_call return self._make_api_call(operation_name, kwargs) File "/usr/local/lib/python3.10/dist-packages/botocore/client.py", line 933, in _make_api_call handler, event_response = self.meta.events.emit_until_response( File "/usr/local/lib/python3.10/dist-packages/botocore/hooks.py", line 416, in emit_until_response return self._emitter.emit_until_response(aliased_event_name, **kwargs) File "/usr/local/lib/python3.10/dist-packages/botocore/hooks.py", line 271, in emit_until_response responses = self._emit(event_name, kwargs, stop_on_response=True) File "/usr/local/lib/python3.10/dist-packages/botocore/hooks.py", line 239, in _emit response = handler(**kwargs) File "/usr/local/lib/python3.10/dist-packages/botocore/utils.py", line 3088, in conditionally_calculate_md5 md5_digest = calculate_md5(body, **kwargs) File "/usr/local/lib/python3.10/dist-packages/botocore/utils.py", line 3055, in calculate_md5 binary_md5 = _calculate_md5_from_file(body) File "/usr/local/lib/python3.10/dist-packages/botocore/utils.py", line 3068, in _calculate_md5_from_file md5.update(chunk) TypeError: Strings must be encoded before hashing
Solution:
You need to open the file in binary mode ("rb"
) . Instead of
with open("testtext.txt", "r") as f:
use
with open("testtext.txt", "rb") as f:
This will fix the issue.