AWS S3 synced files transformed into hex filess3cmd put only files that do not exist in destination bucketI'm trying to use “aws s3 sync” on my EC2 instance. Is the '--exclude' option broken?Backing up data stored on Amazon S3Move files from ftp server to s3Server incremental backup to AWS GlacierUnable to run aws s3 sync as a cron jobAWS CloudFormation - Calling CLI from Init can't get credentialsOpsworks Stack and S3 access, can't get files.How to create an OFFLINE Incremental backup of an AWS S3 bucketServer access logging on AWS S3
60s or 70s novel about Empire of Man making 1st contact with 1st discovered alien race
Longest bridge/tunnel that can be cycled over/through?
Excel division by 0 error when trying to average results of formulas
Is there a DSLR/mirorless camera with minimal options like a classic, simple SLR?
What are neighboring ports?
What aircraft was used as Air Force One for the flight between Southampton and Shannon?
What are some really overused phrases in French that are common nowadays?
What is the color of artificial intelligence?
The Frozen Wastes
If I leave the US through an airport, do I have to return through the same airport?
How can I make 12 tone and atonal melodies sound interesting?
Which languages would be most useful in Europe at the end of the 19th century?
Understanding "Current Draw" in terms of "Ohm's Law"
Someone whose aspirations exceed abilities or means
Can a human be transformed into a Mind Flayer?
Is it expected that a reader will skip parts of what you write?
What is exactly Avijja -- and how to uproot it?
How to communicate to my GM that not being allowed to use stealth isn't fun for me?
Why was this person allowed to become Grand Maester?
Should I put programming books I wrote a few years ago on my resume?
Why can I traceroute to this IP address, but not ping?
Is it safe to change the harddrive power feature so that it never turns off?
Proving that a Russian cryptographic standard is too structured
Why not invest in precious metals?
AWS S3 synced files transformed into hex files
s3cmd put only files that do not exist in destination bucketI'm trying to use “aws s3 sync” on my EC2 instance. Is the '--exclude' option broken?Backing up data stored on Amazon S3Move files from ftp server to s3Server incremental backup to AWS GlacierUnable to run aws s3 sync as a cron jobAWS CloudFormation - Calling CLI from Init can't get credentialsOpsworks Stack and S3 access, can't get files.How to create an OFFLINE Incremental backup of an AWS S3 bucketServer access logging on AWS S3
.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty height:90px;width:728px;box-sizing:border-box;
I used AWS cli to sync my S3 bucket to my computer before deleting the bucket. I did not notice it right after the sync but I now realized that most of my files have been turned into hex files (?). For example, a CSS file I had now looks like:
1f8b 0800 632b 405b 02ff cd5c 4d93 e4b8
71bd fb57 9477 439e 1945 574f 83f5 5d1d
b256 b643 111b b10a 1d56 071d 7c01 49b0
0ad3 24c1 01c1 eeae e998 ff6e 8044 b2c0
ea07 7a1d 7bb1 0eab 69e0 0104 8187 4466
e2b1 3eff f15f ff65 f1c7 c5e2 afaa 368b
bfbc 8856 5562 b1be 4fee 1f16 e965 f153
ce9f c589 d7f9 65b1 5c9c 8d69 8e9f 3f17
and so on.
Not all of my files have been transformed but there does not seem to be any obvious pattern to which files have been transformed. The file sizes seem more or less consistent with original uncorrupted files from what I remember.
Is this my files being corrupted while syncing or is there something else going on? Can I do something to recover my files? I do not have access to the original S3 bucket anymore.
amazon-web-services amazon-s3 aws-cli
asked May 24 at 15:38
tripatheeatripatheea
132
add a comment |
I used AWS cli to sync my S3 bucket to my computer before deleting the bucket. I did not notice it right after the sync but I now realized that most of my files have been turned into hex files (?). For example, a CSS file I had now looks like:
1f8b 0800 632b 405b 02ff cd5c 4d93 e4b8
71bd fb57 9477 439e 1945 574f 83f5 5d1d
b256 b643 111b b10a 1d56 071d 7c01 49b0
0ad3 24c1 01c1 eeae e998 ff6e 8044 b2c0
ea07 7a1d 7bb1 0eab 69e0 0104 8187 4466
e2b1 3eff f15f ff65 f1c7 c5e2 afaa 368b
bfbc 8856 5562 b1be 4fee 1f16 e965 f153
ce9f c589 d7f9 65b1 5c9c 8d69 8e9f 3f17
and so on.
Not all of my files have been transformed but there does not seem to be any obvious pattern to which files have been transformed. The file sizes seem more or less consistent with original uncorrupted files from what I remember.
Is this my files being corrupted while syncing or is there something else going on? Can I do something to recover my files? I do not have access to the original S3 bucket anymore.
amazon-web-services amazon-s3 aws-cli
asked May 24 at 15:38
tripatheeatripatheea
132
add a comment |
I used AWS cli to sync my S3 bucket to my computer before deleting the bucket. I did not notice it right after the sync but I now realized that most of my files have been turned into hex files (?). For example, a CSS file I had now looks like:
1f8b 0800 632b 405b 02ff cd5c 4d93 e4b8
71bd fb57 9477 439e 1945 574f 83f5 5d1d
b256 b643 111b b10a 1d56 071d 7c01 49b0
0ad3 24c1 01c1 eeae e998 ff6e 8044 b2c0
ea07 7a1d 7bb1 0eab 69e0 0104 8187 4466
e2b1 3eff f15f ff65 f1c7 c5e2 afaa 368b
bfbc 8856 5562 b1be 4fee 1f16 e965 f153
ce9f c589 d7f9 65b1 5c9c 8d69 8e9f 3f17
and so on.
Not all of my files have been transformed but there does not seem to be any obvious pattern to which files have been transformed. The file sizes seem more or less consistent with original uncorrupted files from what I remember.
Is this my files being corrupted while syncing or is there something else going on? Can I do something to recover my files? I do not have access to the original S3 bucket anymore.
amazon-web-services amazon-s3 aws-cli
asked May 24 at 15:38
tripatheeatripatheea
132
I used AWS cli to sync my S3 bucket to my computer before deleting the bucket. I did not notice it right after the sync but I now realized that most of my files have been turned into hex files (?). For example, a CSS file I had now looks like:
1f8b 0800 632b 405b 02ff cd5c 4d93 e4b8
71bd fb57 9477 439e 1945 574f 83f5 5d1d
b256 b643 111b b10a 1d56 071d 7c01 49b0
0ad3 24c1 01c1 eeae e998 ff6e 8044 b2c0
ea07 7a1d 7bb1 0eab 69e0 0104 8187 4466
e2b1 3eff f15f ff65 f1c7 c5e2 afaa 368b
bfbc 8856 5562 b1be 4fee 1f16 e965 f153
ce9f c589 d7f9 65b1 5c9c 8d69 8e9f 3f17
and so on.
Not all of my files have been transformed but there does not seem to be any obvious pattern to which files have been transformed. The file sizes seem more or less consistent with original uncorrupted files from what I remember.
Is this my files being corrupted while syncing or is there something else going on? Can I do something to recover my files? I do not have access to the original S3 bucket anymore.
amazon-web-services amazon-s3 aws-cli
amazon-web-services amazon-s3 aws-cli
asked May 24 at 15:38
tripatheeatripatheea
132
asked May 24 at 15:38
tripatheeatripatheea
132
asked May 24 at 15:38
tripatheeatripatheea
132
asked May 24 at 15:38
tripatheeatripatheea
132
asked May 24 at 15:38
tripatheeatripatheea
132
132
add a comment |
add a comment |
1 Answer
1
active
oldest
votes
The first two bytes of your file, 0x1f 0x8b, are the magic number signifying the gzip file compression format. You likely had stored these files, gzipped, in the bucket so that they could be downloaded by browsers in their compressed format, using less bandwidth, and they have been synced to your computer exactly as they existed in S3.
To verify, make a copy in a safe place, then -- working only on the copy, not the original -- rename one test file, e.g. foo.css to foo.css.gz, then run gzip -d foo.css.gz. This should leave you with a file called foo.css with content that is in a more recognizable form.
answered May 25 at 21:06
Michael - sqlbotMichael - sqlbot
16.6k3563
add a comment |
Your Answer
StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "2"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);
else
createEditor();
);
function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);
);
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fserverfault.com%2fquestions%2f968738%2faws-s3-synced-files-transformed-into-hex-files%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
The first two bytes of your file, 0x1f 0x8b, are the magic number signifying the gzip file compression format. You likely had stored these files, gzipped, in the bucket so that they could be downloaded by browsers in their compressed format, using less bandwidth, and they have been synced to your computer exactly as they existed in S3.
To verify, make a copy in a safe place, then -- working only on the copy, not the original -- rename one test file, e.g. foo.css to foo.css.gz, then run gzip -d foo.css.gz. This should leave you with a file called foo.css with content that is in a more recognizable form.
answered May 25 at 21:06
Michael - sqlbotMichael - sqlbot
16.6k3563
add a comment |
The first two bytes of your file, 0x1f 0x8b, are the magic number signifying the gzip file compression format. You likely had stored these files, gzipped, in the bucket so that they could be downloaded by browsers in their compressed format, using less bandwidth, and they have been synced to your computer exactly as they existed in S3.
To verify, make a copy in a safe place, then -- working only on the copy, not the original -- rename one test file, e.g. foo.css to foo.css.gz, then run gzip -d foo.css.gz. This should leave you with a file called foo.css with content that is in a more recognizable form.
answered May 25 at 21:06
Michael - sqlbotMichael - sqlbot
16.6k3563
add a comment |
The first two bytes of your file, 0x1f 0x8b, are the magic number signifying the gzip file compression format. You likely had stored these files, gzipped, in the bucket so that they could be downloaded by browsers in their compressed format, using less bandwidth, and they have been synced to your computer exactly as they existed in S3.
To verify, make a copy in a safe place, then -- working only on the copy, not the original -- rename one test file, e.g. foo.css to foo.css.gz, then run gzip -d foo.css.gz. This should leave you with a file called foo.css with content that is in a more recognizable form.
answered May 25 at 21:06
Michael - sqlbotMichael - sqlbot
16.6k3563
The first two bytes of your file, 0x1f 0x8b, are the magic number signifying the gzip file compression format. You likely had stored these files, gzipped, in the bucket so that they could be downloaded by browsers in their compressed format, using less bandwidth, and they have been synced to your computer exactly as they existed in S3.
To verify, make a copy in a safe place, then -- working only on the copy, not the original -- rename one test file, e.g. foo.css to foo.css.gz, then run gzip -d foo.css.gz. This should leave you with a file called foo.css with content that is in a more recognizable form.
answered May 25 at 21:06
Michael - sqlbotMichael - sqlbot
16.6k3563
answered May 25 at 21:06
Michael - sqlbotMichael - sqlbot
16.6k3563
answered May 25 at 21:06
Michael - sqlbotMichael - sqlbot
16.6k3563
answered May 25 at 21:06
Michael - sqlbotMichael - sqlbot
16.6k3563
16.6k3563
add a comment |
add a comment |
Thanks for contributing an answer to Server Fault!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fserverfault.com%2fquestions%2f968738%2faws-s3-synced-files-transformed-into-hex-files%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
