Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Provides updates to S3 Java files. #6955

Merged
merged 10 commits into from
Oct 9, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 8 additions & 4 deletions .doc_gen/metadata/s3-control_metadata.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -24,9 +24,15 @@ s3-control_CreateJob:
github: javav2/example_code/s3
sdkguide:
excerpts:
- description:
- description: Create an asynchronous S3 job.
snippet_tags:
- s3control.java2.create_job.async.main
- description: Create a compliance retention job.
snippet_tags:
- s3control.java2.create_job.compliance.main
- description: Create a legal hold off job.
snippet_tags:
- s3control.java2.create_job.compliance.main
services:
s3-control: {CreateJob}
s3-control_PutJobTagging:
Expand Down Expand Up @@ -108,10 +114,8 @@ s3-control_UpdateJobPriority:
services:
s3-control: {UpdateJobPriority}
s3-control_Basics:
title: Learn core operations for'&S3Control;' using an &AWS; SDK
title_abbrev: Learn core operations
synopsis: learn core operations for'&S3Control;'.
category: Scenarios
category: Basics
languages:
Java:
versions:
Expand Down
14 changes: 7 additions & 7 deletions javav2/example_code/s3/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,29 +45,29 @@ Code examples that show you how to perform the essential operations within a ser

Code excerpts that show you how to call individual service functions.

- [CopyObject](src/main/java/com/example/s3/scenario/S3Actions.java#L203)
- [CreateBucket](src/main/java/com/example/s3/scenario/S3Actions.java#L85)
- [DeleteBucket](src/main/java/com/example/s3/scenario/S3Actions.java#L342)
- [CopyObject](src/main/java/com/example/s3/scenario/S3Actions.java#L212)
- [CreateBucket](src/main/java/com/example/s3/scenario/S3Actions.java#L90)
- [DeleteBucket](src/main/java/com/example/s3/scenario/S3Actions.java#L353)
- [DeleteBucketPolicy](src/main/java/com/example/s3/DeleteBucketPolicy.java#L6)
- [DeleteBucketWebsite](src/main/java/com/example/s3/DeleteWebsiteConfiguration.java#L6)
- [DeleteObject](src/main/java/com/example/s3/scenario/S3Actions.java#L315)
- [DeleteObject](src/main/java/com/example/s3/scenario/S3Actions.java#L325)
- [DeleteObjects](src/main/java/com/example/s3/DeleteMultiObjects.java#L6)
- [GetBucketAcl](src/main/java/com/example/s3/GetAcl.java#L6)
- [GetBucketPolicy](src/main/java/com/example/s3/GetBucketPolicy.java#L6)
- [GetObject](src/main/java/com/example/s3/scenario/S3Actions.java#L144)
- [GetObject](src/main/java/com/example/s3/scenario/S3Actions.java#L151)
- [GetObjectLegalHold](src/main/java/com/example/s3/lockscenario/S3LockActions.java#L94)
- [GetObjectLockConfiguration](src/main/java/com/example/s3/lockscenario/S3LockActions.java#L345)
- [GetObjectRetention](src/main/java/com/example/s3/lockscenario/S3LockActions.java#L309)
- [HeadObject](src/main/java/com/example/s3/GetObjectContentType.java#L6)
- [ListBuckets](src/main/java/com/example/s3/ListBuckets.java#L6)
- [ListMultipartUploads](src/main/java/com/example/s3/ListMultipartUploads.java#L6)
- [ListObjectsV2](src/main/java/com/example/s3/scenario/S3Actions.java#L177)
- [ListObjectsV2](src/main/java/com/example/s3/scenario/S3Actions.java#L185)
- [PutBucketAcl](src/main/java/com/example/s3/SetAcl.java#L6)
- [PutBucketCors](src/main/java/com/example/s3/S3Cors.java#L6)
- [PutBucketLifecycleConfiguration](src/main/java/com/example/s3/LifecycleConfiguration.java#L6)
- [PutBucketPolicy](src/main/java/com/example/s3/SetBucketPolicy.java#L6)
- [PutBucketWebsite](src/main/java/com/example/s3/SetWebsiteConfiguration.java#L6)
- [PutObject](src/main/java/com/example/s3/scenario/S3Actions.java#L120)
- [PutObject](src/main/java/com/example/s3/scenario/S3Actions.java#L126)
- [PutObjectLegalHold](src/main/java/com/example/s3/lockscenario/S3LockActions.java#L255)
- [PutObjectLockConfiguration](src/main/java/com/example/s3/lockscenario/S3LockActions.java#L206)
- [PutObjectRetention](src/main/java/com/example/s3/lockscenario/S3LockActions.java#L61)
Expand Down
4 changes: 4 additions & 0 deletions javav2/example_code/s3/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -182,6 +182,10 @@
<groupId>software.amazon.awssdk</groupId>
<artifactId>iam-policy-builder</artifactId>
</dependency>
<dependency>
<groupId>software.amazon.awssdk</groupId>
<artifactId>iam</artifactId>
</dependency>
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-core</artifactId>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,7 @@
*/

public class AbortMultipartUploadExamples {
static final String bucketName = "x-" + UUID.randomUUID();
static final String bucketName = "amzn-s3-demo-bucket" + UUID.randomUUID(); // Change bucket name.
static final String key = UUID.randomUUID().toString();
static final String classPathFilePath = "/multipartUploadFiles/s3-userguide.pdf";
static final String filePath = getFullFilePath(classPathFilePath);
Expand All @@ -78,22 +78,28 @@ public static void doAbortIncompleteMultipartUploadsFromList() {
}

// snippet-start:[s3.java2.abort_upload_from_list]
/**
* Aborts all incomplete multipart uploads from the specified S3 bucket.
* <p>
* This method retrieves a list of all incomplete multipart uploads in the specified S3 bucket,
* and then aborts each of those uploads.
*/
public static void abortIncompleteMultipartUploadsFromList() {
ListMultipartUploadsRequest listMultipartUploadsRequest = ListMultipartUploadsRequest.builder()
.bucket(bucketName)
.build();
.bucket(bucketName)
.build();

ListMultipartUploadsResponse response = s3Client.listMultipartUploads(listMultipartUploadsRequest);
List<MultipartUpload> uploads = response.uploads();

AbortMultipartUploadRequest abortMultipartUploadRequest;
for (MultipartUpload upload : uploads) {
abortMultipartUploadRequest = AbortMultipartUploadRequest.builder()
.bucket(bucketName)
.key(upload.key())
.expectedBucketOwner(accountId)
.uploadId(upload.uploadId())
.build();
.bucket(bucketName)
.key(upload.key())
.expectedBucketOwner(accountId)
.uploadId(upload.uploadId())
.build();

AbortMultipartUploadResponse abortMultipartUploadResponse = s3Client.abortMultipartUpload(abortMultipartUploadRequest);
if (abortMultipartUploadResponse.sdkHttpResponse().isSuccessful()) {
Expand All @@ -114,8 +120,8 @@ static void doAbortIncompleteMultipartUploadsOlderThan() {
// snippet-start:[s3.java2.abort_upload_older_than]
static void abortIncompleteMultipartUploadsOlderThan(Instant pointInTime) {
ListMultipartUploadsRequest listMultipartUploadsRequest = ListMultipartUploadsRequest.builder()
.bucket(bucketName)
.build();
.bucket(bucketName)
.build();

ListMultipartUploadsResponse response = s3Client.listMultipartUploads(listMultipartUploadsRequest);
List<MultipartUpload> uploads = response.uploads();
Expand All @@ -125,11 +131,11 @@ static void abortIncompleteMultipartUploadsOlderThan(Instant pointInTime) {
logger.info("Found multipartUpload with upload ID [{}], initiated [{}]", upload.uploadId(), upload.initiated());
if (upload.initiated().isBefore(pointInTime)) {
abortMultipartUploadRequest = AbortMultipartUploadRequest.builder()
.bucket(bucketName)
.key(upload.key())
.expectedBucketOwner(accountId)
.uploadId(upload.uploadId())
.build();
.bucket(bucketName)
.key(upload.key())
.expectedBucketOwner(accountId)
.uploadId(upload.uploadId())
.build();

AbortMultipartUploadResponse abortMultipartUploadResponse = s3Client.abortMultipartUpload(abortMultipartUploadRequest);
if (abortMultipartUploadResponse.sdkHttpResponse().isSuccessful()) {
Expand All @@ -156,9 +162,9 @@ static void doAbortMultipartUploadUsingUploadId() {
static void abortMultipartUploadUsingUploadId() {
String uploadId = startUploadReturningUploadId();
AbortMultipartUploadResponse response = s3Client.abortMultipartUpload(b -> b
.uploadId(uploadId)
.bucket(bucketName)
.key(key));
.uploadId(uploadId)
.bucket(bucketName)
.key(key));

if (response.sdkHttpResponse().isSuccessful()) {
logger.info("Upload ID [{}] to bucket [{}] successfully aborted.", uploadId, bucketName);
Expand All @@ -181,16 +187,16 @@ static void doAbortMultipartUploadsUsingLifecycleConfig() {
// snippet-start:[s3.java2.abort_upload_using_lifecycle_config]
static void abortMultipartUploadsUsingLifecycleConfig() {
Collection<LifecycleRule> lifeCycleRules = List.of(LifecycleRule.builder()
.abortIncompleteMultipartUpload(b -> b.
daysAfterInitiation(7))
.status("Enabled")
.filter(SdkBuilder::build) // Filter element is required.
.build());
.abortIncompleteMultipartUpload(b -> b.
daysAfterInitiation(7))
.status("Enabled")
.filter(SdkBuilder::build) // Filter element is required.
.build());

// If the action is successful, the service sends back an HTTP 200 response with an empty HTTP body.
PutBucketLifecycleConfigurationResponse response = s3Client.putBucketLifecycleConfiguration(b -> b
.bucket(bucketName)
.lifecycleConfiguration(b1 -> b1.rules(lifeCycleRules)));
.bucket(bucketName)
.lifecycleConfiguration(b1 -> b1.rules(lifeCycleRules)));

if (response.sdkHttpResponse().isSuccessful()) {
logger.info("Rule to abort incomplete multipart uploads added to bucket.");
Expand Down Expand Up @@ -243,8 +249,8 @@ static void doMultipartUpload() {

static String step1CreateMultipartUpload() {
CreateMultipartUploadResponse createMultipartUploadResponse = s3Client.createMultipartUpload(b -> b
.bucket(bucketName)
.key(key));
.bucket(bucketName)
.key(key));
return createMultipartUploadResponse.uploadId();
}

Expand All @@ -262,20 +268,20 @@ static List<CompletedPart> step2UploadParts(String uploadId) {

bb.flip(); // Swap position and limit before reading from the buffer.
UploadPartRequest uploadPartRequest = UploadPartRequest.builder()
.bucket(bucketName)
.key(key)
.uploadId(uploadId)
.partNumber(partNumber)
.build();
.bucket(bucketName)
.key(key)
.uploadId(uploadId)
.partNumber(partNumber)
.build();

UploadPartResponse partResponse = s3Client.uploadPart(
uploadPartRequest,
RequestBody.fromByteBuffer(bb));
uploadPartRequest,
RequestBody.fromByteBuffer(bb));

CompletedPart part = CompletedPart.builder()
.partNumber(partNumber)
.eTag(partResponse.eTag())
.build();
.partNumber(partNumber)
.eTag(partResponse.eTag())
.build();
completedParts.add(part);
logger.info("Part {} upload", partNumber);

Expand All @@ -292,10 +298,10 @@ static List<CompletedPart> step2UploadParts(String uploadId) {

static void step3CompleteMultipartUpload(String uploadId, List<CompletedPart> completedParts) {
s3Client.completeMultipartUpload(b -> b
.bucket(bucketName)
.key(key)
.uploadId(uploadId)
.multipartUpload(CompletedMultipartUpload.builder().parts(completedParts).build()));
.bucket(bucketName)
.key(key)
.uploadId(uploadId)
.multipartUpload(CompletedMultipartUpload.builder().parts(completedParts).build()));
}

static String startUploadReturningUploadId() {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@
// snippet-start:[s3.java2.basicOpsWithChecksums.full]
public class BasicOpsWithChecksums {
static final S3Client s3Client = S3Client.create();
static final String bucketName = "x-" + UUID.randomUUID();
static final String bucketName = "amzn-s3-demo-bucket" + UUID.randomUUID(); // Change bucket name.
static final String key = UUID.randomUUID().toString();
private static final Logger logger = LoggerFactory.getLogger(BasicOpsWithChecksums.class);

Expand Down Expand Up @@ -105,7 +105,7 @@ public void putObjectWithChecksum() {
.bucket(bucketName)
.key(key)
.checksumAlgorithm(ChecksumAlgorithm.CRC32),
RequestBody.fromString("This is a test"));
RequestBody.fromString("This is a test"));
}
// snippet-end:[s3.java2.basicOpsWithChecksums.putObject]

Expand All @@ -115,7 +115,7 @@ public GetObjectResponse getObjectWithChecksum() {
.bucket(bucketName)
.key(key)
.checksumMode(ChecksumMode.ENABLED))
.response();
.response();
}
// snippet-end:[s3.java2.basicOpsWithChecksums.getObject]

Expand All @@ -127,20 +127,20 @@ public void putObjectWithPrecalculatedChecksum(String filePath) {
.bucket(bucketName)
.key(key)
.checksumSHA256(checksum)),
RequestBody.fromFile(Paths.get(filePath)));
RequestBody.fromFile(Paths.get(filePath)));
}
// snippet-end:[s3.java2.basicOpsWithChecksums.putObjectPreCalc]

// snippet-start:[s3.java2.basicOpsWithChecksums.multiPartTm]
public void multipartUploadWithChecksumTm(String filePath) {
S3TransferManager transferManager = S3TransferManager.create();
UploadFileRequest uploadFileRequest = UploadFileRequest.builder()
.putObjectRequest(b -> b
.bucket(bucketName)
.key(key)
.checksumAlgorithm(ChecksumAlgorithm.SHA1))
.source(Paths.get(filePath))
.build();
.putObjectRequest(b -> b
.bucket(bucketName)
.key(key)
.checksumAlgorithm(ChecksumAlgorithm.SHA1))
.source(Paths.get(filePath))
.build();
FileUpload fileUpload = transferManager.uploadFile(uploadFileRequest);
fileUpload.completionFuture().join();
transferManager.close();
Expand All @@ -153,9 +153,9 @@ public void multipartUploadWithChecksumS3Client(String filePath) {

// Initiate the multipart upload.
CreateMultipartUploadResponse createMultipartUploadResponse = s3Client.createMultipartUpload(b -> b
.bucket(bucketName)
.key(key)
.checksumAlgorithm(algorithm)); // Checksum specified on initiation.
.bucket(bucketName)
.key(key)
.checksumAlgorithm(algorithm)); // Checksum specified on initiation.
String uploadId = createMultipartUploadResponse.uploadId();

// Upload the parts of the file.
Expand All @@ -172,22 +172,22 @@ public void multipartUploadWithChecksumS3Client(String filePath) {

bb.flip(); // Swap position and limit before reading from the buffer.
UploadPartRequest uploadPartRequest = UploadPartRequest.builder()
.bucket(bucketName)
.key(key)
.uploadId(uploadId)
.checksumAlgorithm(algorithm) // Checksum specified on each part.
.partNumber(partNumber)
.build();
.bucket(bucketName)
.key(key)
.uploadId(uploadId)
.checksumAlgorithm(algorithm) // Checksum specified on each part.
.partNumber(partNumber)
.build();

UploadPartResponse partResponse = s3Client.uploadPart(
uploadPartRequest,
RequestBody.fromByteBuffer(bb));
uploadPartRequest,
RequestBody.fromByteBuffer(bb));

CompletedPart part = CompletedPart.builder()
.partNumber(partNumber)
.checksumCRC32(partResponse.checksumCRC32()) // Provide the calculated checksum.
.eTag(partResponse.eTag())
.build();
.partNumber(partNumber)
.checksumCRC32(partResponse.checksumCRC32()) // Provide the calculated checksum.
.eTag(partResponse.eTag())
.build();
completedParts.add(part);

bb.clear();
Expand All @@ -200,10 +200,10 @@ public void multipartUploadWithChecksumS3Client(String filePath) {

// Complete the multipart upload.
s3Client.completeMultipartUpload(b -> b
.bucket(bucketName)
.key(key)
.uploadId(uploadId)
.multipartUpload(CompletedMultipartUpload.builder().parts(completedParts).build()));
.bucket(bucketName)
.key(key)
.uploadId(uploadId)
.multipartUpload(CompletedMultipartUpload.builder().parts(completedParts).build()));
}
// snippet-end:[s3.java2.basicOpsWithChecksums.multiPartS3Client]

Expand Down
Loading
Loading