Skip to content
This repository has been archived by the owner on Jul 2, 2024. It is now read-only.

Commit

Permalink
Merge pull request #61 from jdeathe/develop
Browse files Browse the repository at this point in the history
Release changes for 1.6.0
  • Loading branch information
jdeathe authored Jan 23, 2023
2 parents 03c904c + f220eae commit c336602
Show file tree
Hide file tree
Showing 16 changed files with 772 additions and 88 deletions.
69 changes: 48 additions & 21 deletions AWS_VM_IMPORT.md
Original file line number Diff line number Diff line change
@@ -1,12 +1,8 @@
# Importing an OVA VM Image to an AWS EC2 AMI

## Prerequisites
- [Install](http://docs.aws.amazon.com/cli/latest/userguide/installing.html) and [configure](http://docs.aws.amazon.com/cli/latest/userguide/cli-config-files.html) the AWS CLI.
```
# On OSX
$ easy_install awscli
```
- Follow the guidance in [VMImportPrerequisites](http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/VMImportPrerequisites.html).
- [Install](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html) and [configure](https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-files.html) the AWS CLI.
- Follow the guidance in [VM Import/Export Prerequisites](https://docs.aws.amazon.com/vm-import/latest/userguide/vmimport-image-import.html#import-image-prereqs).
- For non-root IAM users you may need to attach the following inline policy to the `vmimport` service role.
```
{
Expand Down Expand Up @@ -39,29 +35,54 @@ $ aws s3 mb \
--region eu-west-1
```

Upload the ova image to the S3 bucket `{{bucket_name}}`.
### Upload an Image

Upload the `vmdk`,`ova` or `raw` format image to the S3 bucket `{{bucket_name}}`.

```
$ aws s3 cp \
builds/CentOS-7.7.1908-x86_64-Minimal-AMI-en_US.ova \
s3://{{bucket_name}}/CentOS-7.7.1908-x86_64-Minimal-AMI-en_US.ova
builds/CentOS-7.9.2009-x86_64-Minimal-AMI-en_US.vmdk \
s3://{{bucket_name}}/CentOS-7.9.2009-x86_64-Minimal-AMI-en_US.vmdk
```

<!-- ### Copy an Image Between Reigional Buckets
```
$ aws s3 cp \
s3://{{source_bucket_name}}/CentOS-7.9.2009-x86_64-Minimal-AMI-en_US.vmdk \
s3://{{target_bucket_name}}/CentOS-7.9.2009-x86_64-Minimal-AMI-en_US.vmdk
``` -->

### Import

Import the image from the S3 bucket `{{bucket_name}}`.

```

This will return some output that includes an ImportTaskId. e.g. `import-ami-08e9c882d14e44b83`.

<!-- ```
$ aws ec2 import-image \
--region eu-west-1 \
--cli-input-json '{ "LicenseType": "BYOL", "Description": "jdeathe/CentOS-7.7.1908-x86_64-Minimal-AMI-en_US", "DiskContainers": [ { "Description": "jdeathe/CentOS-7.7.1908-x86_64-Minimal-AMI-en_US", "Format": "ova", "UserBucket": { "S3Bucket": "{{bucket_name}}", "S3Key" : "CentOS-7.7.1908-x86_64-Minimal-AMI-en_US.ova" } } ]}'
--cli-input-json '{ "LicenseType": "BYOL", "Description": "jdeathe/CentOS-7.9.2009-x86_64-Minimal-AMI-en_US", "DiskContainers": [ { "Description": "jdeathe/CentOS-7.9.2009-x86_64-Minimal-AMI-en_US", "Format": "vmdk", "UserBucket": { "S3Bucket": "{{bucket_name}}", "S3Key" : "CentOS-7.9.2009-x86_64-Minimal-AMI-en_US.vmdk" } } ]}'
``` -->

```
$ aws ec2 import-image \
--architecture x86_64 \
--boot-mode legacy-bios \
--description "jdeathe/CentOS-7.9.2009-x86_64-Minimal-AMI-en_US" \
--disk-containers '[ { "Description": "jdeathe/CentOS-7.9.2009-x86_64-Minimal-AMI-en_US", "Format": "raw", "UserBucket": { "S3Bucket": "{{bucket_name}}", "S3Key" : "CentOS-7.9.2009-x86_64-Minimal-AMI-en_US.vmdk" } } ]' \
--license-type BYOL \
--platform Linux \
--region eu-west-1
```

Check progress

```
$ aws ec2 describe-import-image-tasks \
--region eu-west-1
--region eu-west-1 \
--import-task-ids {{ImportTaskId}}
```

If you need to cancel the import:
Expand All @@ -74,19 +95,22 @@ $ aws ec2 cancel-import-task \
Once complete, identify the `ImageId`.

```
$ aws ec2 describe-images \
$ aws --output json ec2 describe-images \
--owners self \
--region eu-west-1
--region eu-west-1 \
--filters '[ { "Name": "description", "Values": ["AWS-VMImport service*"] } ]' \
| grep '"ImageId"' \
| sed -r -e 's~^(.*"ImageId": ")(ami-[0-9a-zA-Z]+)(",)$~\2~'
```

Create copy of the intermediate image with required name and description values.
Create copy of the intermediate image with required name and description values to the target region(s).

```
$ aws ec2 copy-image \
--source-region eu-west-1 \
--region eu-west-1 \
--name "jdeathe/centos-7-x86_64-minimal-cloud-init-en_us-v7.7.0" \
--description "CentOS-7.7.1908 x86_64 Minimal Cloud-Init Base Image - en_US locale" \
--name "jdeathe/centos-7-x86_64-minimal-en_us-v7.7.0" \
--description "CentOS-7.9.2009 x86_64 Minimal Base Image - en_US locale" \
--source-image-id {{ImageId}}
```

Expand All @@ -103,9 +127,12 @@ $ aws ec2 deregister-image \
Identify SpanshotId. i.e. With a Description starting "Created by AWS-VMImport service".

```
$ aws ec2 describe-snapshots \
$ aws --output json ec2 describe-snapshots \
--region eu-west-1 \
--owner-ids self
--owner-ids self \
--filters '[ { "Name": "description", "Values": ["Created by AWS-VMImport service*"] } ]' \
| grep '"SnapshotId"' \
| sed -r -e 's~^(.*"SnapshotId": ")(snap-[0-9a-zA-Z]+)(",)$~\2~'
```

Remove snapshot created by the import process.
Expand All @@ -116,9 +143,9 @@ $ aws ec2 delete-snapshot \
--snapshot-id {{SnapshotId}}
```

Remove the source OVA image from the S3 bucket.
Remove the source image from the S3 bucket.

```
$ aws s3 rm \
s3://{{bucket_name}}/CentOS-7.7.1908-x86_64-Minimal-AMI-en_US.ova
s3://{{bucket_name}}/CentOS-7.9.2009-x86_64-Minimal-AMI-en_US.vmdk
```
174 changes: 174 additions & 0 deletions CentOS-7-Minimal-AMI-libvirt.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,174 @@
{
"variables": {
"build_accelerator": "kvm",
"build_export_format": "qcow2",
"build_iso_checksum": "07b94e6b1a0b0260b94c83d6bb76b26bf7a310dc78d7a9c7432809fb9bc6194a",
"build_iso_checksum_type": "sha256",
"build_iso_filename": "CentOS-7-x86_64-Minimal-2009.iso",
"build_iso_target_path": "isos/x86_64",
"build_name": "CentOS-7.9.2009-x86_64-Minimal-AMI-en_US",
"build_output_directory": "builds",
"build_output_format": "vmdk",
"guest_boot_timeout": "0",
"guest_bootloader_append": "net.ifnames=0 biosdevname=0",
"guest_cpus": "1",
"guest_firewall_disabled": "false",
"guest_hard_disk_size": "4096",
"guest_keyboard": "us",
"guest_language": "en_US.UTF-8",
"guest_lv_root_fstype": "ext4",
"guest_lv_root_mkfsoptions": "-i 4096",
"guest_lv_root_size": "1024",
"guest_lv_swap_size": "0",
"guest_name": "CentOS-7.9.2009-x86_64-Minimal-AMI-en_US",
"guest_partition_boot_fstype": "ext4",
"guest_partition_boot_size": "1024",
"guest_memory": "1024",
"guest_selinux": "permissive",
"guest_timezone": "Etc/UTC",
"guest_vram": "16",
"guest_vg_root_reserved_space": "0",
"ssh_root_password": "centos",
"ssh_user": "centos",
"ssh_user_authorized_keys": "",
"ssh_user_home": "/home/centos",
"ssh_user_password": "centos",
"ssh_user_shell": "/bin/bash",
"ssh_user_sudo": "ALL=(ALL) NOPASSWD:ALL"
},
"builders": [
{
"type": "qemu",
"name": "{{user `build_name`}}",
"accelerator": "{{user `build_accelerator`}}",
"boot_command": [
"<esc><wait>",
"linux ks=http://{{.HTTPIP}}:{{.HTTPPort}}/centos-7-minimal.cfg",
" BOOT_TIMEOUT={{user `guest_boot_timeout`}}",
" BOOTLOADER_APPEND=\"{{user `guest_bootloader_append`}}\"",
" BOOTLOADER_THEME=details",
" LANG={{user `guest_language`}}",
" KEYTABLE={{user `guest_keyboard`}}",
" TIMEZONE={{user `guest_timezone`}}",
" ROOTPW={{user `ssh_root_password`}}",
" FIREWALL_DISABLED={{user `guest_firewall_disabled`}}",
" SELINUX={{user `guest_selinux`}}",
" LV_ROOT_FSTYPE={{user `guest_lv_root_fstype`}}",
" LV_ROOT_MKFSOPTIONS=\"{{user `guest_lv_root_mkfsoptions`}}\"",
" LV_ROOT_SIZE={{user `guest_lv_root_size`}}",
" LV_SWAP_SIZE={{user `guest_lv_swap_size`}}",
" PART_BOOT_FSTYPE={{user `guest_partition_boot_fstype`}}",
" PART_BOOT_SIZE={{user `guest_partition_boot_size`}}",
" VG_ROOT_RESERVED_SPACE={{user `guest_vg_root_reserved_space`}}",
"<enter><wait>"
],
"boot_key_interval": "10ms",
"boot_wait": "3s",
"disk_interface": "virtio-scsi",
"disk_discard": "unmap",
"disk_detect_zeroes": "on",
"disk_cache": "unsafe",
"disk_image": false,
"disk_compression": true,
"disk_size": "{{user `guest_hard_disk_size`}}",
"format": "{{user `build_export_format`}}",
"headless": true,
"http_directory": "http",
"iso_checksum": "{{user `build_iso_checksum`}}",
"iso_target_path": "{{user `build_iso_target_path`}}",
"iso_url": "{{user `build_iso_target_path`}}/{{user `build_iso_filename`}}",
"machine_type": "q35",
"memory": "{{user `guest_memory`}}",
"net_device": "virtio-net",
"qemuargs": [
[ "-global", "virtio-pci.disable-modern=on" ],
[ "-cpu", "host" ]
],
"ssh_username": "root",
"ssh_password": "{{user `ssh_root_password`}}",
"ssh_port": 22,
"ssh_wait_timeout": "10m",
"shutdown_command": "/sbin/shutdown --no-wall -P now",
"shutdown_timeout": "1m",
"vm_name": "{{user `guest_name`}}"
}
],
"post-processors": [
{
"type": "compress",
"output": "{{user `build_output_directory`}}/{{.BuildName}}.{{user `build_export_format`}}.tar.gz",
"compression_level": 9,
"keep_input_artifact": true
},
{
"type": "shell-local",
"execute_command": "chmod +x \"{{.Script}}\"; {{.Vars}} /bin/bash \"{{.Script}}\" {{user `build_output_directory`}} {{user `build_export_format`}} {{user `build_output_format`}}",
"inline_shebang": "/bin/bash -e",
"inline": [
"if [[ -n ${1} ]] && [[ -n ${2} ]] && [[ -n ${PACKER_BUILD_NAME} ]] && [[ -d output-${PACKER_BUILD_NAME} ]]",
"then",
" echo '--> Moving build artifact.'",
" mv -f output-${PACKER_BUILD_NAME}/${PACKER_BUILD_NAME} ${1}/${PACKER_BUILD_NAME}.${2}",
" printf -- '--> Convert build artifact from %s to %s.\n' ${2} ${3}",
" if [[ ${3} == vmdk ]]",
" then",
" qemu-img convert -f ${2} -O ${3} -o subformat=streamOptimized,compat6 ${1}/${PACKER_BUILD_NAME}.${2} ${1}/${PACKER_BUILD_NAME}.${3}",
" else",
" qemu-img convert -f ${2} -O ${3} ${1}/${PACKER_BUILD_NAME}.${2} ${1}/${PACKER_BUILD_NAME}.${3}",
" fi",
"fi",
"echo '--> Cleanup output directories.'",
"find . -mindepth 1 -maxdepth 1 -type d -name \"output-*\" -exec rm -rf '{}' +"
]
}
],
"provisioners": [
{
"type": "file",
"source": "src/etc/systemd/system/grow-root.service",
"destination": "/etc/systemd/system/grow-root.service"
},
{
"type": "shell",
"execute_command": "chmod +x \"{{.Path}}\"; env {{.Vars}} /bin/bash \"{{.Path}}\"",
"remote_folder": "/var/tmp",
"skip_clean": false,
"environment_vars": [
"GUEST_LANG={{user `guest_language`}}",
"SSH_USER={{user `ssh_user`}}",
"SSH_USER_AUTHORIZED_KEYS={{user `ssh_user_authorized_keys`}}",
"SSH_USER_HOME={{user `ssh_user_home`}}",
"SSH_USER_PASSWORD={{user `ssh_user_password`}}",
"SSH_USER_SHELL={{user `ssh_user_shell`}}",
"SSH_USER_SUDO={{user `ssh_user_sudo`}}"
],
"scripts": [
"scripts/install/nfs-utils.sh",
"scripts/install/tuned-virtual-guest.sh",
"scripts/install/cloud-init.sh",
"scripts/install/cloud-utils-growpart.sh",
"scripts/install/systemd-unit-file-grow-root.sh",
"scripts/cloud-init/disable-locale-module.sh",
"scripts/cloud-init/add-datasource-list-aws.sh",
"scripts/cloud-init/add-logging-output.sh",
"scripts/cloud-init/add-preserve-hostname.sh",
"scripts/common/ifcfg-name-to-device.sh",
"scripts/common/sshd-config-non-root-key-auth.sh",
"scripts/common/sudoers-default-not-requiretty.sh",
"scripts/common/ssh-user.sh",
"scripts/common/seal-virtual-guest.sh",
"scripts/common/locale-trim-definitions.sh",
"scripts/common/locale-trim-translations.sh",
"scripts/common/locale-lock-sysconfig.sh",
"scripts/common/purge-documentation.sh",
"scripts/common/purge-desktop-graphics.sh",
"scripts/common/purge-temporary-directories.sh",
"scripts/common/yum-cleanup.sh",
"scripts/common/rpm-rebuild-db.sh",
"scripts/common/stop-and-truncate-logs.sh",
"scripts/common/zero-out-disks.sh",
"scripts/common/lock-root-user.sh"
]
}
]
}
30 changes: 16 additions & 14 deletions CentOS-7-Minimal-AMI-virtualbox.json
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
{
"variables": {
"build_export_format": "ova",
"build_iso_checksum": "9a2c47d97b9975452f7d582264e9fc16d108ed8252ac6816239a3b58cef5c53d",
"build_iso_checksum": "07b94e6b1a0b0260b94c83d6bb76b26bf7a310dc78d7a9c7432809fb9bc6194a",
"build_iso_checksum_type": "sha256",
"build_iso_filename": "CentOS-7-x86_64-Minimal-1908.iso",
"build_iso_filename": "CentOS-7-x86_64-Minimal-2009.iso",
"build_iso_target_path": "isos/x86_64",
"build_name": "CentOS-7.7.1908-x86_64-Minimal-AMI-en_US",
"build_name": "CentOS-7.9.2009-x86_64-Minimal-AMI-en_US",
"build_output_directory": "builds",
"guest_boot_timeout": "0",
"guest_bootloader_append": "net.ifnames=0 biosdevname=0",
Expand All @@ -18,7 +18,7 @@
"guest_lv_root_mkfsoptions": "-i 4096",
"guest_lv_root_size": "1024",
"guest_lv_swap_size": "0",
"guest_name": "CentOS-7.7.1908-x86_64-Minimal-AMI-en_US",
"guest_name": "CentOS-7.9.2009-x86_64-Minimal-AMI-en_US",
"guest_partition_boot_fstype": "ext4",
"guest_partition_boot_size": "1024",
"guest_memory": "1024",
Expand Down Expand Up @@ -59,6 +59,7 @@
" VG_ROOT_RESERVED_SPACE={{user `guest_vg_root_reserved_space`}}",
"<enter><wait>"
],
"boot_keygroup_interval": "500ms",
"boot_wait": "3s",
"disk_size": "{{user `guest_hard_disk_size`}}",
"format": "{{user `build_export_format`}}",
Expand All @@ -68,7 +69,6 @@
"headless": true,
"http_directory": "http",
"iso_checksum": "{{user `build_iso_checksum`}}",
"iso_checksum_type": "{{user `build_iso_checksum_type`}}",
"iso_interface": "sata",
"iso_target_path": "{{user `build_iso_target_path`}}",
"iso_url": "{{user `build_iso_target_path`}}/{{user `build_iso_filename`}}",
Expand All @@ -79,20 +79,22 @@
"shutdown_command": "/sbin/shutdown --no-wall -P now",
"shutdown_timeout": "1m",
"vboxmanage": [
["modifyvm","{{.Name}}","--cpus","1"],
["modifyvm","{{.Name}}","--memory","1024"],
["modifyvm","{{.Name}}","--vram","8"],
["modifyvm","{{.Name}}","--hpet","on"],
["modifyvm","{{.Name}}","--accelerate3d","off"],
["modifyvm","{{.Name}}","--accelerate2dvideo","off"],
["modifyvm","{{.Name}}","--accelerate3d","off"],
["modifyvm","{{.Name}}","--audio","none"],
["modifyvm","{{.Name}}","--biosbootmenu","disabled"],
["modifyvm","{{.Name}}","--bioslogodisplaytime","0"],
["modifyvm","{{.Name}}","--bioslogofadein","off"],
["modifyvm","{{.Name}}","--bioslogofadeout","off"],
["modifyvm","{{.Name}}","--bioslogodisplaytime","0"],
["modifyvm","{{.Name}}","--biosbootmenu","disabled"],
["modifyvm","{{.Name}}","--clipboard","disabled"],
["modifyvm","{{.Name}}","--cpus","1"],
["modifyvm","{{.Name}}","--draganddrop","disabled"],
["modifyvm","{{.Name}}","--graphicscontroller","vmsvga"],
["modifyvm","{{.Name}}","--hpet","on"],
["modifyvm","{{.Name}}","--memory","1024"],
["modifyvm","{{.Name}}","--nat-localhostreachable1", "on"],
["modifyvm","{{.Name}}","--usb","off"],
["modifyvm","{{.Name}}","--audio","none"]
["modifyvm","{{.Name}}","--vram","10"]
],
"vboxmanage_post": [
["modifyvm","{{.Name}}","--cpus","{{user `guest_cpus`}}"],
Expand Down Expand Up @@ -120,7 +122,7 @@
"inline": [
"if [[ -n ${1} ]] && [[ -n ${2} ]] && [[ -n ${PACKER_BUILD_NAME} ]] && [[ -d output-${PACKER_BUILD_NAME} ]]; then",
" echo '--> Moving build artifact.'",
" mv -f output-${PACKER_BUILD_NAME}/${PACKER_BUILD_NAME}.${2} ${1}/${PACKER_BUILD_NAME}.${2}",
" mv -f output-${PACKER_BUILD_NAME}/${PACKER_BUILD_NAME} ${1}/${PACKER_BUILD_NAME}.${2}",
"fi",
"echo '--> Cleanup output directories.'",
"find . -mindepth 1 -maxdepth 1 -type d -name \"output-*\" -exec rm -rf '{}' +"
Expand Down
Loading

0 comments on commit c336602

Please sign in to comment.