Search posts, tags, users, and pages
I appreciate your post. Thank you.
I have one concern:
It seems like half of the UserData bash script should be part of the CFN code instead. Specifically these steps should all be in CFN so that CloudFormation can track the resources, no?:
# get instance-id
# associate the Elastic IP address to the instance
# attach volume
# wait for volume to be properly attached
# format the volume if doesn't have file system
# create mount root folder
# mount the volume to folder
That's in UserData to make the service self-healing.
In case the instance becomes unhealthy and Autoscaling terminates it and starts a new one, the new instance will be attached the Elastic IP and EBS volume automatically.
Stefan Olaru Oh, I didn't think of that. It wasn't clear to me from the AWS autoscaling docs what happens with the attached EBS (when set to not delete) upon autoscaling. That's very tricky. Thank you for the quick response.
Stefan Olaru May I also ask why you generate the admin api key using a lambda function instead of using secrets manager?
That’s what I had at hand at time of writing, but as you say, doing it with GenerateSecretString can also work and it’s a more elegant solution ;) I would appreciate if you can share how you end up solving it.
Stefan Olaru Certainly!
Stefan Olaru Took a me a while (converted to cdk and changed/added some things), but here is what I did to generate the bootstrap api key and startup the typesense instance:
Have AWS Secretsmanager generate the bootstrap api key without it ever touching my laptop:
typesense_bootstrap_api_key = secretsmanager.Secret(self, "TypesenseBootstrapAPIKeySecret",
secret_name="TypesenseBootstrapAPIKey",
generate_secret_string=secretsmanager.SecretStringGenerator(
exclude_uppercase=True,
exclude_punctuation=True,
password_length=64,
),
)
Get the Bootstrap API Key from SecretsManager upon execution of the UserData:
sudo yum install jq
# get the api key for bootstrapping typesense
secretsmanager_resp=$(aws secretsmanager get-secret-value \
--secret-id {typesense_bootstrap_api_key.secret_name})
ApiKey=$(echo $secretsmanager_resp | jq -r .SecretString)
The Typesense EC2 Instance IAM Role needs additional permissions to get this secret from SecretsManager. I created a SecretsManagerReadOnlyPolicy and attached it to the role:
secrets_manager_read_only_policy = iam.ManagedPolicy(self, "SecretsManagerReadOnlyPolicy",
managed_policy_name="SecretsManagerReadOnly",
statements=[iam.PolicyStatement(
effect=iam.Effect.ALLOW,
actions=[
"secretsmanager:GetResourcePolicy",
"secretsmanager:GetSecretValue",
"secretsmanager:DescribeSecret",
"secretsmanager:ListSecretVersionIds"
],
resources=["*"]
)]
)
I do end up using a lambda function to generate api keys. It gets the bootstrap or admin key (depending on what keyname you pass in the payload) from secretsmanager, reaches out to typesense to generate a new key with whatever name and permissions were passed in the payload, and then stores that new key in secretsmanager. Hopefully this can be run to generate the basic keys I'd want (admin, readwrite, readonly, idk what else) upon CDK deploy, but I don't have a great way to ensure it runs after the typesense server actually starts up for the first time without doing something very hacky.
By the way, my UserData script needed this addition to get the EC2_INSTANCE_ID:
# get instance-id
TOKEN=$(curl -X PUT -H "X-aws-ec2-metadata-token-ttl-seconds: 600" "instance-data/latest/api/token")
EC2_INSTANCE_ID=$(curl -H "X-aws-ec2-metadata-token: $TOKEN" -s instance-data/latest/meta-data/instance-id)
This is all built into a greater cdk project I have going, but I will pull this out and put it in a public repo when I have time.
Calvin Butler Awesome, thanks for the detailed reply! 👏