3-tier app: EC2 se EC2 communication

Pehla block kaam ka folder banata hai, Amazon Linux 2023 ki latest AMI ID nikaalta hai, aur ek chhota tag function banata hai jo har resource pe Name aur Project tag lagane ka text tayyar karta hai.

Uske baad shopkart-vpc (10.60.0.0/16) banta hai aur usme DNS hostnames chalu kiye jaate hain. Phir do subnets bante hain: public (10.60.1.0/24) aur private (10.60.2.0/24). Public subnet mein launch hone wale servers ko apne aap public IP milta hai.

Phir ek internet gateway banake VPC se joda jaata hai. Public route table mein "0.0.0.0/0 internet gateway se" wala route daal ke use public subnet se joda jaata hai.

Aakhir mein public subnet mein ek Elastic IP ke saath NAT Gateway banta hai, aur uske available hone ka intezaar kiya jaata hai. Private route table mein "0.0.0.0/0 NAT se" wala route daala jaata hai, taaki private servers bahar ja sakein lekin bahar se koi andar na aa sake.

export AWS_DEFAULT_REGION=us-east-1 TYPE=t3.micro
mkdir -p ~/deepdive/scn1 && cd ~/deepdive/scn1
AMI_ID=$(aws ssm get-parameter --name /aws/service/ami-amazon-linux-latest/al2023-ami-kernel-default-x86_64 --query Parameter.Value --output text)
tag() { echo "ResourceType=$1,Tags=[{Key=Name,Value=$2},{Key=Project,Value=deepdive-scn1}]"; }

VPC=$(aws ec2 create-vpc --cidr-block 10.60.0.0/16 --tag-specifications "$(tag vpc shopkart-vpc)" --query Vpc.VpcId --output text)
aws ec2 modify-vpc-attribute --vpc-id $VPC --enable-dns-hostnames
PUB=$(aws ec2 create-subnet --vpc-id $VPC --cidr-block 10.60.1.0/24 --availability-zone us-east-1a --tag-specifications "$(tag subnet shopkart-public)" --query Subnet.SubnetId --output text)
PRIV=$(aws ec2 create-subnet --vpc-id $VPC --cidr-block 10.60.2.0/24 --availability-zone us-east-1a --tag-specifications "$(tag subnet shopkart-private)" --query Subnet.SubnetId --output text)
aws ec2 modify-subnet-attribute --subnet-id $PUB --map-public-ip-on-launch

IGW=$(aws ec2 create-internet-gateway --query InternetGateway.InternetGatewayId --output text)
aws ec2 attach-internet-gateway --internet-gateway-id $IGW --vpc-id $VPC
PUB_RT=$(aws ec2 create-route-table --vpc-id $VPC --query RouteTable.RouteTableId --output text)
aws ec2 create-route --route-table-id $PUB_RT --destination-cidr-block 0.0.0.0/0 --gateway-id $IGW >/dev/null
aws ec2 associate-route-table --route-table-id $PUB_RT --subnet-id $PUB >/dev/null

EIP=$(aws ec2 allocate-address --query AllocationId --output text)
NAT=$(aws ec2 create-nat-gateway --subnet-id $PUB --allocation-id $EIP --query NatGateway.NatGatewayId --output text)
aws ec2 wait nat-gateway-available --nat-gateway-ids $NAT
PRIV_RT=$(aws ec2 create-route-table --vpc-id $VPC --query RouteTable.RouteTableId --output text)
aws ec2 create-route --route-table-id $PRIV_RT --destination-cidr-block 0.0.0.0/0 --nat-gateway-id $NAT >/dev/null
aws ec2 associate-route-table --route-table-id $PRIV_RT --subnet-id $PRIV >/dev/null
echo "VPC=$VPC PUB=$PUB PRIV=$PRIV NAT=$NAT"

PL-065 course se — asli console reference

VPC create, Internet Gateway aur NAT Gateway banane ka wahi console — is scenario mein subnets/CIDR tumhare apne honge.

Yeh block teeno tiers ke liye teen security groups banata hai aur unhe ek zanjeer (chain) mein jodta hai. Web SG pe port 80 poori duniya ke liye khula hai. App SG pe port 8080 sirf un servers ke liye khula hai jinpe web SG laga ho, aur DB SG pe port 3306 sirf un servers ke liye jinpe app SG laga ho. Source mein IP ki jagah security group dene ka fayda yeh hai ki servers ke IP badal jaayein, tab bhi rules sahi rehte hain. Kisi bhi SG mein port 22 nahi khola gaya hai.

sg() { aws ec2 create-security-group --vpc-id $VPC --group-name $1 --description "$2" --query GroupId --output text; }
WEB_SG=$(sg shopkart-web-sg "web tier")
APP_SG=$(sg shopkart-app-sg "app tier")
DB_SG=$(sg shopkart-db-sg "db tier")
aws ec2 authorize-security-group-ingress --group-id $WEB_SG --protocol tcp --port 80   --cidr 0.0.0.0/0 >/dev/null
aws ec2 authorize-security-group-ingress --group-id $APP_SG --protocol tcp --port 8080 --source-group $WEB_SG >/dev/null
aws ec2 authorize-security-group-ingress --group-id $DB_SG  --protocol tcp --port 3306 --source-group $APP_SG >/dev/null

PL-065 course se — asli console reference

Security group inbound rules tab — source ki jagah tum yahan doosre SG ka reference doge.

Pehle db-userdata.sh banaya jaata hai. Yeh server ke pehli baar boot hone pe MariaDB install karke chalu karta hai. Phir shop database aur ek app user banata hai, products table banata hai aur usme teen products daal deta hai.

Uske baad ek launch function banaya jaata hai. Yeh diye gaye naam, subnet, security group aur user-data ke saath instance launch karta hai; har instance pe Lab 1 wala role lagata hai aur IMDSv2 zaroori karta hai. Function instance ID aur private IP lautata hai.

Aakhir mein DB server bina public IP ke private subnet mein launch hota hai, aur uska ID aur IP DB_ID aur DB_IP mein rakh liye jaate hain.

cat > db-userdata.sh <<'EOF'
#!/bin/bash
dnf install -y mariadb105-server
systemctl enable --now mariadb
mysql -e "
CREATE DATABASE shop;
CREATE USER 'app'@'10.60.%' IDENTIFIED BY 'AppPass123!';
GRANT ALL ON shop.* TO 'app'@'10.60.%';
CREATE TABLE shop.products (id INT PRIMARY KEY AUTO_INCREMENT, name VARCHAR(50), price INT);
INSERT INTO shop.products (name, price) VALUES ('Keyboard', 1499), ('Mouse', 699), ('Monitor', 8999);"
EOF
launch() {  # name subnet sg userdata [extra args]
  aws ec2 run-instances --image-id $AMI_ID --instance-type $TYPE --subnet-id $2 --security-group-ids $3 \
    --iam-instance-profile Name=deepdive-ec2-role --user-data file://$4 --metadata-options HttpTokens=required \
    --tag-specifications "$(tag instance $1)" "${@:5}" --query 'Instances[0].[InstanceId,PrivateIpAddress]' --output text
}
read DB_ID DB_IP <<< "$(launch shopkart-db $PRIV $DB_SG db-userdata.sh --no-associate-public-ip-address)"
echo "DB $DB_ID $DB_IP"

Dhyan do ki DB user sirf 10.60.% se, yaani sirf isi VPC ke andar se, login kar sakta hai. Is tarah database ke level pe bhi least privilege lagaya gaya hai.

Yeh block app server ka user-data banata hai. Bahar wale heredoc mein EOF bina quotes ke hai, isliye shell $DB_IP ki jagah DB ka asli IP bhar deta hai. Andar ke 'PY' aur 'UNIT' quotes mein hain, taaki server pe unke andar kuch na badle.

Server pe yeh script pymysql install karti hai aur /opt/app/app.py likhti hai. Yeh ek chhota HTTP server hai jo port 8080 pe do raaste deta hai: /health aur /products. /products DB se products padh ke JSON mein lautata hai, aur DB tak na pahunch paaye toh error ko JSON mein dikhata hai.

Phir ek systemd service banti hai, taaki app reboot ke baad bhi apne aap chalu ho aur crash hone pe dobara start ho.

Launch se pehle grep se check kiya jaata hai ki DB ka IP file mein sahi se bhar gaya hai. Uske baad app server bhi bina public IP ke private subnet mein launch hota hai.

cat > app-userdata.sh <<EOF
#!/bin/bash
dnf install -y python3-pip
pip3 install pymysql
mkdir -p /opt/app
cat > /opt/app/app.py <<'PY'
import json
import socket
from http.server import BaseHTTPRequestHandler, HTTPServer

import pymysql

DB_HOST = "$DB_IP"


class Handler(BaseHTTPRequestHandler):
    def do_GET(self):
        if self.path == "/health":
            return self.send_json(200, {"ok": True, "host": socket.gethostname()})
        if self.path == "/products":
            try:
                conn = pymysql.connect(host=DB_HOST, user="app", password="AppPass123!",
                                       database="shop", connect_timeout=3)
                with conn.cursor() as cur:
                    cur.execute("SELECT id, name, price FROM products")
                    rows = [{"id": r[0], "name": r[1], "price": r[2]} for r in cur.fetchall()]
                conn.close()
                return self.send_json(200, {"served_by": socket.gethostname(), "products": rows})
            except Exception as exc:
                return self.send_json(500, {"error": str(exc)})
        self.send_json(404, {"error": "not found"})

    def send_json(self, code, body):
        data = json.dumps(body).encode()
        self.send_response(code)
        self.send_header("Content-Type", "application/json")
        self.send_header("Content-Length", str(len(data)))
        self.end_headers()
        self.wfile.write(data)


HTTPServer(("0.0.0.0", 8080), Handler).serve_forever()
PY
cat > /etc/systemd/system/shop-app.service <<'UNIT'
[Unit]
Description=ShopKart API
After=network-online.target
[Service]
ExecStart=/usr/bin/python3 /opt/app/app.py
Restart=always
[Install]
WantedBy=multi-user.target
UNIT
systemctl daemon-reload
systemctl enable --now shop-app
EOF
grep DB_HOST app-userdata.sh        # DB ka IP bhara hua dikhna chahiye
read APP_ID APP_IP <<< "$(launch shopkart-app $PRIV $APP_SG app-userdata.sh --no-associate-public-ip-address)"
echo "APP $APP_ID $APP_IP"

Yeh block web server ka user-data banata hai. Isme nginx install hota hai aur ek config likhi jaati hai. Config ke hisaab se /api/ se shuru hone wali har request app server ke port 8080 pe bhej di jaati hai (yahan bhi $APP_IP ki jagah asli IP bhar jaata hai), aur baaki saari requests pe seedha "ShopKart web tier OK" lauta diya jaata hai.

Web server public subnet mein launch hota hai, isliye use public IP milta hai. Phir teeno servers ke chalu hone ka intezaar kiya jaata hai, web server ka public IP nikaala jaata hai, aur browser mein kholne wala URL print kiya jaata hai.

cat > web-userdata.sh <<EOF
#!/bin/bash
dnf install -y nginx
cat > /etc/nginx/conf.d/shopkart.conf <<'CONF'
server {
    listen 80;
    server_name shopkart;
    location /api/ {
        proxy_pass http://$APP_IP:8080/;
        proxy_connect_timeout 3s;
    }
    location / {
        default_type text/plain;
        return 200 "ShopKart web tier OK\n";
    }
}
CONF
systemctl enable --now nginx
EOF
read WEB_ID WEB_PRIV <<< "$(launch shopkart-web $PUB $WEB_SG web-userdata.sh)"
aws ec2 wait instance-running --instance-ids $DB_ID $APP_ID $WEB_ID
WEB_IP=$(aws ec2 describe-instances --instance-ids $WEB_ID --query 'Reservations[0].Instances[0].PublicIpAddress' --output text)
echo "open: http://$WEB_IP/api/products   (2-3 min baad)"

nginx ka apna default server bhi port 80 pe hota hai. conf.d wala config pehle load hota hai, isliye aam taur pe wahi default ban jaata hai. Agar phir bhi nginx ka default welcome page dikhe, toh /etc/nginx/nginx.conf mein se default server {} block hata do aur systemctl reload nginx chalao.