Elysia Tools

在一个清晰的工作台里浏览工具、案例、可视化和主题指南。

热门分类

专题

语言
ZHDERUPTENFRES

Elysia Tools

导航

云服务

AWS S3 Lambda 示例

AWS S3和Lambda无服务器架构示例,包括S3触发器、Lambda函数和无服务器应用模式

案例

这个案例集合中的条目

配置简单

S3 Lambda 基础触发器

基本的S3存储桶配置,包含用于处理文件上传和修改的Lambda触发器

难度

4/10

预计时间

20 min

标签

aws, s3, lambda, serverless, infrastructure

前置条件

AWS account, AWS CLI, SAM CLI, Node.js

config
# AWS S3 and Lambda Basic Configuration
# Serverless architecture for file processing and automation

# 1. AWS SAM Template (template.yaml)
AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: S3 Lambda file processing application

Globals:
  Function:
    Runtime: nodejs18.x
    Timeout: 300
    MemorySize: 512
    Environment:
      Variables:
        TABLE_NAME: !Ref FileProcessingTable

Resources:
  # S3 Bucket for file uploads
  FileUploadBucket:
    Type: AWS::S3::Bucket
    Properties:
      BucketName: !Sub 'file-upload-bucket-${AWS::StackName}'
      PublicAccessBlockConfiguration:
        BlockPublicAcls: true
        BlockPublicPolicy: true
        IgnorePublicAcls: true
        RestrictPublicBuckets: true
      BucketEncryption:
        ServerSideEncryptionConfiguration:
          - ServerSideEncryptionByDefault:
              SSEAlgorithm: AES256
      NotificationConfiguration:
        LambdaConfigurations:
          - Event: s3:ObjectCreated:*
            Function: !GetAtt FileProcessorFunction.Arn

  # Lambda function for processing uploaded files
  FileProcessorFunction:
    Type: AWS::Serverless::Function
    Properties:
      CodeUri: lambda/
      Handler: index.handler
      Runtime: nodejs18.x
      Environment:
        Variables:
          BUCKET_NAME: !Ref FileUploadBucket
          TABLE_NAME: !Ref FileProcessingTable
      Policies:
        - S3ReadPolicy:
            BucketName: !Ref FileUploadBucket
        - DynamoDBCrudPolicy:
            TableName: !Ref FileProcessingTable

  # DynamoDB table for tracking file processing
  FileProcessingTable:
    Type: AWS::DynamoDB::Table
    Properties:
      TableName: !Sub 'file-processing-table-${AWS::StackName}'
      AttributeDefinitions:
        - AttributeName: fileId
          AttributeType: S
      KeySchema:
        - AttributeName: fileId
          KeyType: HASH
      BillingMode: PAY_PER_REQUEST
      StreamSpecification:
        StreamViewType: NEW_AND_OLD_IMAGES

  # IAM Role for Lambda with S3 and DynamoDB permissions
  LambdaExecutionRole:
    Type: AWS::IAM::Role
    Properties:
      RoleName: !Sub 's3-lambda-execution-role-${AWS::StackName}'
      AssumeRolePolicyDocument:
        Version: '2012-10-17'
        Statement:
          - Effect: Allow
            Principal:
              Service: lambda.amazonaws.com
            Action: sts:AssumeRole
      ManagedPolicyArns:
        - arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole
      Policies:
        - PolicyName: S3AccessPolicy
          PolicyDocument:
            Version: '2012-10-17'
            Statement:
              - Effect: Allow
                Action:
                  - s3:GetObject
                  - s3:PutObject
                  - s3:DeleteObject
                Resource: !Sub 'arn:aws:s3:::${FileUploadBucket}/*'
        - PolicyName: DynamoDBAccessPolicy
          PolicyDocument:
            Version: '2012-10-17'
            Statement:
              - Effect: Allow
                Action:
                  - dynamodb:PutItem
                  - dynamodb:GetItem
                  - dynamodb:UpdateItem
                  - dynamodb:Query
                  - dynamodb:Scan
                Resource:
                  - !GetAtt FileProcessingTable.Arn
                  - !Sub '${FileProcessingTable.Arn}/index/*'

Outputs:
  BucketName:
    Description: S3 bucket for file uploads
    Value: !Ref FileUploadBucket
    Export:
      Name: !Sub '${AWS::StackName}-FileUploadBucket'

  FunctionName:
    Description: Lambda function ARN
    Value: !GetAtt FileProcessorFunction.Arn
    Export:
      Name: !Sub '${AWS::StackName}-FileProcessorFunction'

  TableName:
    Description: DynamoDB table name
    Value: !Ref FileProcessingTable
    Export:
      Name: !Sub '${AWS::StackName}-FileProcessingTable'
代码javascript中等

Lambda文件处理函数

Node.js Lambda函数,用于处理S3文件上传,包含元数据提取和缩略图生成

难度

6/10

预计时间

35 min

标签

aws, lambda, s3, image-processing, serverless

前置条件

Node.js, AWS SDK, Sharp library, DynamoDB

javascript
// Lambda Function for S3 File Processing
// File: lambda/index.js

const AWS = require('aws-sdk');
const sharp = require('sharp');
const { S3Client, GetObjectCommand, PutObjectCommand } = require('@aws-sdk/client-s3');
const { DynamoDBClient, PutItemCommand, UpdateItemCommand } = require('@aws-sdk/client-dynamodb');
const { marshall, unmarshall } = require('@aws-sdk/util-dynamodb');

// Initialize AWS SDK clients
const s3Client = new S3Client({ region: process.env.AWS_REGION });
const dynamoDbClient = new DynamoDBClient({ region: process.env.AWS_REGION });

// Environment variables
const BUCKET_NAME = process.env.BUCKET_NAME;
const TABLE_NAME = process.env.TABLE_NAME;
const THUMBNAIL_FOLDER = 'thumbnails/';

// Supported image formats for processing
const SUPPORTED_FORMATS = ['jpg', 'jpeg', 'png', 'webp', 'gif'];
const THUMBNAIL_SIZES = [150, 300, 600];

/**
 * Main Lambda handler for S3 events
 */
exports.handler = async (event, context) => {
  console.log('Received S3 event:', JSON.stringify(event, null, 2));

  try {
    // Process each record in the S3 event
    for (const record of event.Records) {
      await processS3Record(record);
    }

    return {
      statusCode: 200,
      body: JSON.stringify({
        message: 'Files processed successfully',
        recordsProcessed: event.Records.length
      })
    };
  } catch (error) {
    console.error('Error processing files:', error);

    return {
      statusCode: 500,
      body: JSON.stringify({
        error: 'File processing failed',
        details: error.message
      })
    };
  }
};

/**
 * Process individual S3 record
 */
async function processS3Record(record) {
  const eventName = record.eventName;
  const bucketName = record.s3.bucket.name;
  const objectKey = decodeURIComponent(record.s3.object.key.replace(/\+/g, ' '));

  console.log(`Processing ${eventName} for object: ${objectKey} in bucket: ${bucketName}`);

  // Skip processing for thumbnail files to prevent infinite loops
  if (objectKey.startsWith(THUMBNAIL_FOLDER)) {
    console.log('Skipping thumbnail file');
    return;
  }

  // Extract file extension
  const fileExtension = objectKey.split('.').pop().toLowerCase();

  if (SUPPORTED_FORMATS.includes(fileExtension)) {
    console.log(`Processing supported image format: ${fileExtension}`);
    await processImageFile(bucketName, objectKey, fileExtension);
  } else {
    console.log(`File format ${fileExtension} not supported for image processing`);
    await processGenericFile(bucketName, objectKey);
  }
}

/**
 * Process image files - generate thumbnails and extract metadata
 */
async function processImageFile(bucketName, objectKey, fileExtension) {
  const startTime = Date.now();
  const fileId = generateFileId(objectKey);

  try {
    // Get the original image from S3
    const getObjectCommand = new GetObjectCommand({
      Bucket: bucketName,
      Key: objectKey
    });

    const response = await s3Client.send(getObjectCommand);
    const imageBuffer = await streamToBuffer(response.Body);

    // Extract image metadata using sharp
    const metadata = await sharp(imageBuffer).metadata();

    console.log('Image metadata:', metadata);

    // Generate thumbnails in different sizes
    const thumbnails = await generateThumbnails(imageBuffer, objectKey, fileExtension);

    // Store file information in DynamoDB
    const fileRecord = {
      fileId,
      objectKey,
      bucketName,
      originalSize: imageBuffer.length,
      contentType: response.ContentType,
      width: metadata.width,
      height: metadata.height,
      format: metadata.format,
      size: metadata.size,
      thumbnails,
      processingTime: Date.now() - startTime,
      status: 'completed',
      processedAt: new Date().toISOString(),
      expiresAt: new Date(Date.now() + 30 * 24 * 60 * 60 * 1000).toISOString() // 30 days
    };

    await saveFileRecord(fileRecord);

    console.log(`Successfully processed image: ${objectKey}`);
  } catch (error) {
    console.error(`Error processing image ${objectKey}:`, error);

    // Save error record to DynamoDB
    const errorRecord = {
      fileId,
      objectKey,
      bucketName,
      status: 'failed',
      error: error.message,
      processedAt: new Date().toISOString(),
      expiresAt: new Date(Date.now() + 7 * 24 * 60 * 60 * 1000).toISOString() // 7 days for failed records
    };

    await saveFileRecord(errorRecord);
    throw error;
  }
}

/**
 * Process non-image files
 */
async function processGenericFile(bucketName, objectKey) {
  const fileId = generateFileId(objectKey);
  const startTime = Date.now();

  try {
    // Get object metadata
    const getObjectCommand = new GetObjectCommand({
      Bucket: bucketName,
      Key: objectKey
    });

    const response = await s3Client.send(getObjectCommand);

    // Store file information in DynamoDB
    const fileRecord = {
      fileId,
      objectKey,
      bucketName,
      originalSize: response.ContentLength,
      contentType: response.ContentType,
      processingTime: Date.now() - startTime,
      status: 'completed',
      processedAt: new Date().toISOString(),
      expiresAt: new Date(Date.now() + 30 * 24 * 60 * 60 * 1000).toISOString()
    };

    await saveFileRecord(fileRecord);

    console.log(`Successfully processed file: ${objectKey}`);
  } catch (error) {
    console.error(`Error processing file ${objectKey}:`, error);
    throw error;
  }
}

/**
 * Generate thumbnails in multiple sizes
 */
async function generateThumbnails(imageBuffer, originalKey, fileExtension) {
  const thumbnails = [];
  const baseName = originalKey.split('.').slice(0, -1).join('.');

  for (const size of THUMBNAIL_SIZES) {
    try {
      // Generate thumbnail
      const thumbnailBuffer = await sharp(imageBuffer)
        .resize(size, size, {
          fit: 'inside',
          withoutEnlargement: true
        })
        .jpeg({ quality: 80 })
        .toBuffer();

      // Generate thumbnail key
      const thumbnailKey = `${THUMBNAIL_FOLDER}${baseName}_${size}px.${fileExtension}`;

      // Upload thumbnail to S3
      const putObjectCommand = new PutObjectCommand({
        Bucket: BUCKET_NAME,
        Key: thumbnailKey,
        Body: thumbnailBuffer,
        ContentType: 'image/jpeg',
        Metadata: {
          originalKey,
          size: size.toString()
        }
      });

      await s3Client.send(putObjectCommand);

      thumbnails.push({
        size,
        key: thumbnailKey,
        url: `https://${BUCKET_NAME}.s3.${process.env.AWS_REGION}.amazonaws.com/${thumbnailKey}`,
        sizeBytes: thumbnailBuffer.length
      });

      console.log(`Generated thumbnail: ${thumbnailKey} (${size}px)`);
    } catch (error) {
      console.error(`Error generating ${size}px thumbnail:`, error);
    }
  }

  return thumbnails;
}

/**
 * Save file record to DynamoDB
 */
async function saveFileRecord(fileRecord) {
  const putItemCommand = new PutItemCommand({
    TableName: TABLE_NAME,
    Item: marshall(fileRecord)
  });

  await dynamoDbClient.send(putItemCommand);
}

/**
 * Utility function to generate unique file ID
 */
function generateFileId(objectKey) {
  const timestamp = Date.now();
  const randomString = Math.random().toString(36).substring(2, 8);
  const fileHash = Buffer.from(objectKey).toString('base64').substring(0, 8);
  return `${timestamp}-${fileHash}-${randomString}`;
}

/**
 * Convert stream to buffer
 */
async function streamToBuffer(stream) {
  return new Promise((resolve, reject) => {
    const chunks = [];
    stream.on('data', (chunk) => chunks.push(chunk));
    stream.on('error', reject);
    stream.on('end', () => resolve(Buffer.concat(chunks)));
  });
}

// Package.json for Lambda function
/*
{
  "name": "s3-file-processor",
  "version": "1.0.0",
  "description": "Lambda function for processing S3 file uploads",
  "main": "index.js",
  "dependencies": {
    "@aws-sdk/client-s3": "^3.400.0",
    "@aws-sdk/client-dynamodb": "^3.400.0",
    "@aws-sdk/util-dynamodb": "^3.400.0",
    "sharp": "^0.32.6"
  },
  "devDependencies": {
    "aws-sdk": "^2.1490.0"
  }
}
*/
配置复杂

高级无服务器模式

复杂的无服务器架构模式,包括API Gateway集成、事件驱动工作流和监控

难度

8/10

预计时间

60 min

标签

aws, serverless, advanced, architecture, patterns

前置条件

AWS fundamentals, Serverless Framework, Step Functions, EventBridge

config
# Advanced Serverless Architecture Patterns

# 1. Complete Serverless Application with API Gateway, S3, and Lambda
# serverless.yml
service: file-processing-app

frameworkVersion: '3'

provider:
  name: aws
  runtime: nodejs18.x
  region: us-east-1
  stage: ${opt:stage, 'dev'}
  environment:
    STAGE: ${self:provider.stage}
    REGION: ${self:provider.region}
    FILE_TABLE: ${self:service}-files-${self:provider.stage}
    UPLOAD_BUCKET: ${self:service}-uploads-${self:provider.stage}
    THUMBNAIL_BUCKET: ${self:service}-thumbnails-${self:provider.stage}
    PROCESSING_QUEUE_URL: !Ref ProcessingQueue.QueueUrl

  iamRoleStatements:
    - Effect: Allow
      Action:
        - s3:GetObject
        - s3:PutObject
        - s3:DeleteObject
      Resource:
        - !Sub "arn:aws:s3:::${self:provider.environment.UPLOAD_BUCKET}/*"
        - !Sub "arn:aws:s3:::${self:provider.environment.THUMBNAIL_BUCKET}/*"
    - Effect: Allow
      Action:
        - dynamodb:Query
        - dynamodb:Scan
        - dynamodb:GetItem
        - dynamodb:PutItem
        - dynamodb:UpdateItem
        - dynamodb:DeleteItem
      Resource:
        - !GetAtt FilesTable.Arn
        - !Sub "${FilesTable.Arn}/index/*"
    - Effect: Allow
      Action:
        - sqs:SendMessage
        - sqs:ReceiveMessage
        - sqs:DeleteMessage
        - sqs:GetQueueAttributes
      Resource:
        - !GetAtt ProcessingQueue.Arn

functions:
  # API Gateway endpoints
  uploadUrl:
    handler: src/api/uploadUrl.handler
    events:
      - http:
          path: upload/url
          method: post
          cors: true

  fileStatus:
    handler: src/api/fileStatus.handler
    events:
      - http:
          path: files/{fileId}/status
          method: get
          cors: true

  fileList:
    handler: src/api/fileList.handler
    events:
      - http:
          path: files
          method: get
          cors: true

  # S3 trigger for file processing
  processUpload:
    handler: src/processors/processUpload.handler
    events:
      - s3:
          bucket: !Ref UploadBucket
          event: s3:ObjectCreated:*
          rules:
            - prefix: uploads/
          existing: false

  # SQS-based async processing
  thumbnailGenerator:
    handler: src/processors/generateThumbnails.handler
    events:
      - sqs:
          arn:
            !GetAtt ProcessingQueue.Arn
          batchSize: 5
          maximumBatchingWindowInSeconds: 10

  # Scheduled cleanup
  cleanupOldFiles:
    handler: src/jobs/cleanupOldFiles.handler
    events:
      - schedule:
          rate: rate(1 day)
          input:
            cleanupDays: 30

resources:
  Resources:
    UploadBucket:
      Type: AWS::S3::Bucket
      Properties:
        BucketName: ${self:provider.environment.UPLOAD_BUCKET}
        PublicAccessBlockConfiguration:
          BlockPublicAcls: true
          BlockPublicPolicy: true
          IgnorePublicAcls: true
          RestrictPublicBuckets: true
        NotificationConfiguration:
          QueueConfigurations:
            - Event: s3:ObjectCreated:*
              Queue: !GetAtt ProcessingQueue.Arn
              Filter:
                S3Key:
                  Rules:
                    - Name: prefix
                      Value: uploads/

    ThumbnailBucket:
      Type: AWS::S3::Bucket
      Properties:
        BucketName: ${self:provider.environment.THUMBNAIL_BUCKET}
        PublicAccessBlockConfiguration:
          BlockPublicAcls: true
          BlockPublicPolicy: true
          IgnorePublicAcls: true
          RestrictPublicBuckets: true
        CorsConfiguration:
          CorsRules:
            - AllowedOrigins: ['*']
              AllowedHeaders: ['*']
              AllowedMethods: [GET, HEAD]
              MaxAge: 3600

    ProcessingQueue:
      Type: AWS::SQS::Queue
      Properties:
        QueueName: ${self:service}-processing-queue-${self:provider.stage}
        VisibilityTimeout: 900
        MessageRetentionPeriod: 1209600 # 14 days
        RedrivePolicy:
          deadLetterTargetArn: !GetAtt ProcessingDLQ.Arn
          maxReceiveCount: 3

    ProcessingDLQ:
      Type: AWS::SQS::Queue
      Properties:
        QueueName: ${self:service}-processing-dlq-${self:provider.stage}
        MessageRetentionPeriod: 1209600

    FilesTable:
      Type: AWS::DynamoDB::Table
      Properties:
        TableName: ${self:provider.environment.FILE_TABLE}
        BillingMode: PAY_PER_REQUEST
        AttributeDefinitions:
          - AttributeName: fileId
            AttributeType: S
          - AttributeName: userId
            AttributeType: S
          - AttributeName: status
            AttributeType: S
          - AttributeName: createdAt
            AttributeType: N
        KeySchema:
          - AttributeName: fileId
            KeyType: HASH
        GlobalSecondaryIndexes:
          - IndexName: UserStatusIndex
            KeySchema:
              - AttributeName: userId
                KeyType: HASH
              - AttributeName: status
                KeyType: RANGE
            Projection:
              ProjectionType: ALL
          - IndexName: StatusCreatedIndex
            KeySchema:
              - AttributeName: status
                KeyType: HASH
              - AttributeName: createdAt
                KeyType: RANGE
            Projection:
              ProjectionType: ALL
        PointInTimeRecoverySpecification:
          PointInTimeRecoveryEnabled: true
        TTLSpecification:
          AttributeName: expiresAt

    # CloudWatch Alarms
    ProcessingQueueDepthAlarm:
      Type: AWS::CloudWatch::Alarm
      Properties:
        AlarmName: ${self:service}-queue-depth-alarm-${self:provider.stage}
        AlarmDescription: 'Alarm when processing queue depth is too high'
        Namespace: AWS/SQS
        MetricName: ApproximateNumberOfMessagesVisible
        Dimensions:
          - Name: QueueName
            Value: !GetAtt ProcessingQueue.QueueName
        Threshold: 100
        ComparisonOperator: GreaterThanThreshold
        EvaluationPeriods: 2
        Period: 300
        Statistic: Sum
        AlarmActions:
          - !Ref SNSAlertTopic

    # SNS Topic for alerts
    SNSAlertTopic:
      Type: AWS::SNS::Topic
      Properties:
        TopicName: ${self:service}-alerts-${self:provider.stage}

    SNSAlertSubscription:
      Type: AWS::SNS::Subscription
      Properties:
        TopicArn: !Ref SNSAlertTopic
        Protocol: email
        Endpoint: ${ssm:/my-app/alert-email}

# 2. Custom Lambda Authorizer for API Security
# src/auth/customAuthorizer.js
const jwt = require('jsonwebtoken');
const { dynamoDbClient } = require('../lib/aws');

module.exports.handler = async (event) => {
  const token = event.headers.authorization?.replace('Bearer ', '');

  if (!token) {
    return generatePolicy('user', 'Deny', event.methodArn);
  }

  try {
    const decoded = jwt.verify(token, process.env.JWT_SECRET);
    const user = await getUser(decoded.userId);

    if (!user || !user.isActive) {
      return generatePolicy('user', 'Deny', event.methodArn);
    }

    return generatePolicy(user.userId, 'Allow', event.methodArn, {
      userId: user.userId,
      role: user.role,
      permissions: user.permissions
    });
  } catch (error) {
    console.error('Authorization error:', error);
    return generatePolicy('user', 'Deny', event.methodArn);
  }
};

function generatePolicy(principalId, effect, resource, context = {}) {
  return {
    principalId,
    policyDocument: {
      Version: '2012-10-17',
      Statement: [{
        Action: 'execute-api:Invoke',
        Effect: effect,
        Resource: resource
      }]
    },
    context
  };
}

# 3. EventBridge Event Routing
# serverless.yml snippet (add to functions section)
  eventBridgeHandler:
    handler: src/events/eventBridge.handler
    events:
      - eventBridge:
          eventBus: !Ref CustomEventBus
          pattern:
            source:
              - com.myapp.file.processing
            detailType:
              - FileProcessed
              - ProcessingFailed

# 4. Step Functions Workflow for Complex Processing
# step-functions-workflow.json
{
  "Comment": "Complex file processing workflow",
  "StartAt": "ValidateFile",
  "States": {
    "ValidateFile": {
      "Type": "Task",
      "Resource": "arn:aws:lambda:REGION:ACCOUNT_ID:function:validateFile",
      "Next": "CheckFileType",
      "Retry": [
        {
          "ErrorEquals": ["States.ALL"],
          "IntervalSeconds": 2,
          "MaxAttempts": 3,
          "BackoffRate": 2.0
        }
      ]
    },
    "CheckFileType": {
      "Type": "Choice",
      "Choices": [
        {
          "Variable": "$.fileType",
          "StringEquals": "image",
          "Next": "GenerateThumbnails"
        },
        {
          "Variable": "$.fileType",
          "StringEquals": "video",
          "Next": "ExtractVideoMetadata"
        },
        {
          "Variable": "$.fileType",
          "StringEquals": "document",
          "Next": "ExtractText"
        }
      ],
      "Default": "ArchiveFile"
    },
    "GenerateThumbnails": {
      "Type": "Parallel",
      "Branches": [
        {
          "StartAt": "SmallThumbnail",
          "States": {
            "SmallThumbnail": {
              "Type": "Task",
              "Resource": "arn:aws:lambda:REGION:ACCOUNT_ID:function:generateThumbnail",
              "Parameters": {
                "fileKey.$": "$.fileKey",
                "size": 150
              },
              "End": true
            }
          }
        },
        {
          "StartAt": "MediumThumbnail",
          "States": {
            "MediumThumbnail": {
              "Type": "Task",
              "Resource": "arn:aws:lambda:REGION:ACCOUNT_ID:function:generateThumbnail",
              "Parameters": {
                "fileKey.$": "$.fileKey",
                "size": 300
              },
              "End": true
            }
          }
        },
        {
          "StartAt": "LargeThumbnail",
          "States": {
            "LargeThumbnail": {
              "Type": "Task",
              "Resource": "arn:aws:lambda:REGION:ACCOUNT_ID:function:generateThumbnail",
              "Parameters": {
                "fileKey.$": "$.fileKey",
                "size": 600
              },
              "End": true
            }
          }
        }
      ],
      "Next": "UpdateDatabase"
    },
    "ExtractVideoMetadata": {
      "Type": "Task",
      "Resource": "arn:aws:lambda:REGION:ACCOUNT_ID:function:extractVideoMetadata",
      "Next": "UpdateDatabase"
    },
    "ExtractText": {
      "Type": "Task",
      "Resource": "arn:aws:lambda:REGION:ACCOUNT_ID:function:extractText",
      "Next": "UpdateDatabase"
    },
    "UpdateDatabase": {
      "Type": "Task",
      "Resource": "arn:aws:lambda:REGION:ACCOUNT_ID:function:updateDatabase",
      "Next": "SendNotification"
    },
    "ArchiveFile": {
      "Type": "Task",
      "Resource": "arn:aws:lambda:REGION:ACCOUNT_ID:function:archiveFile",
      "Next": "SendNotification"
    },
    "SendNotification": {
      "Type": "Task",
      "Resource": "arn:aws:lambda:REGION:ACCOUNT_ID:function:sendNotification",
      "End": true
    }
  }
}

# 5. Infrastructure as Code with CDK (TypeScript)
# lib/file-processing-stack.ts
import * as cdk from 'aws-cdk-lib';
import { Construct } from 'constructs';
import {
  aws_s3 as s3,
  aws_lambda as lambda,
  aws_sqs as sqs,
  aws_dynamodb as dynamodb,
  aws_apigateway as apigateway,
  aws_events as events,
  aws_events_targets as targets
} from 'aws-cdk-lib';

export class FileProcessingStack extends cdk.Stack {
  constructor(scope: Construct, id: string, props?: cdk.StackProps) {
    super(scope, id, props);

    // S3 Buckets
    const uploadBucket = new s3.Bucket(this, 'UploadBucket', {
      bucketName: 'file-uploads',
      encryption: s3.BucketEncryption.S3_MANAGED,
      blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL,
      removalPolicy: cdk.RemovalPolicy.DESTROY,
    });

    const thumbnailBucket = new s3.Bucket(this, 'ThumbnailBucket', {
      bucketName: 'file-thumbnails',
      encryption: s3.BucketEncryption.S3_MANAGED,
      blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL,
      removalPolicy: cdk.RemovalPolicy.DESTROY,
    });

    // DynamoDB Table
    const filesTable = new dynamodb.Table(this, 'FilesTable', {
      tableName: 'files',
      partitionKey: { name: 'fileId', type: dynamodb.AttributeType.STRING },
      sortKey: { name: 'createdAt', type: dynamodb.AttributeType.NUMBER },
      billingMode: dynamodb.BillingMode.PAY_PER_REQUEST,
      removalPolicy: cdk.RemovalPolicy.DESTROY,
      timeToLiveAttribute: 'expiresAt',
    });

    // Add GSI
    filesTable.addGlobalSecondaryIndex({
      indexName: 'UserIdIndex',
      partitionKey: { name: 'userId', type: dynamodb.AttributeType.STRING },
      sortKey: { name: 'createdAt', type: dynamodb.AttributeType.NUMBER },
    });

    // SQS Queue
    const processingQueue = new sqs.Queue(this, 'ProcessingQueue', {
      queueName: 'file-processing-queue',
      visibilityTimeout: cdk.Duration.minutes(15),
      retentionPeriod: cdk.Duration.days(14),
      deadLetterQueue: {
        maxReceiveCount: 3,
        queue: new sqs.Queue(this, 'ProcessingDLQ', {
          queueName: 'file-processing-dlq',
        }),
      },
    });

    // Lambda Functions
    const fileProcessor = new lambda.Function(this, 'FileProcessor', {
      runtime: lambda.Runtime.NODEJS_18_X,
      handler: 'index.handler',
      code: lambda.Code.fromAsset('lambda/processor'),
      memorySize: 1024,
      timeout: cdk.Duration.minutes(5),
      environment: {
        UPLOAD_BUCKET: uploadBucket.bucketName,
        THUMBNAIL_BUCKET: thumbnailBucket.bucketName,
        FILES_TABLE: filesTable.tableName,
        PROCESSING_QUEUE_URL: processingQueue.queueUrl,
      },
    });

    // Grant permissions
    uploadBucket.grantRead(fileProcessor);
    thumbnailBucket.grantReadWrite(fileProcessor);
    filesTable.grantReadWriteData(fileProcessor);
    processingQueue.grantSendMessages(fileProcessor);

    // EventBridge for event routing
    const eventBus = new events.EventBus(this, 'FileProcessingEventBus', {
      eventBusName: 'file-processing-events',
    });

    // EventBridge rules
    const fileProcessedRule = new events.Rule(this, 'FileProcessedRule', {
      eventBus,
      eventPattern: {
        source: ['com.myapp.file.processor'],
        detailType: ['FileProcessed'],
      },
    });

    fileProcessedRule.addTarget(new targets.LambdaFunction(fileProcessor));
  }
}

工具

经常与该案例搭配使用的工具

相关内容

相关案例集合

相关内容

相关专题

专题

网络基础设施标识符格式检查

在进入配置、清单或数据处理前,检查 IPv4、IPv6、CIDR、MAC 地址、AWS ARN 和 Docker 镜像引用的语法与结构,并明确区分格式证据和可达性、归属、权限及镜像可用性。

专题

药代动力学教学模型与结果核查

用于教学或 QA 的药代公式、模型假设和结果一致性核查,不提供用药剂量、诊断或临床建议。

专题

Excel 公式工程

把公式当代码,配上完整工具链——用常用函数与操作符构建公式、对真实单元格值求值并解释其引用、展开并反向翻译 Excel 365 动态数组公式(LET、LAMBDA、MAP、REDUCE)、把一切重复劳动升级为生成的 VBA 宏。

专题

经典与现代密码工具

在安全的教学或 CTF 场景中比较可逆的经典变换与现代加密,并明确玩具密码不等于安全防护。

首页
工具3212
案例593
可视化471
文件查看器103
专题239
标签333
图像、音频与视频657

图像、音频、视频处理、转换与优化工具

数学、日期与金融647

计算器、数值计算、日期逻辑、统计与金融工具

设计与颜色298

颜色、版式、图形、视觉样式与设计辅助工具

文本与写作209

写作、清洗、格式化、提取与文本分析工具

开发与 Web193

开发者工具、网络处理、Web 调试与自动化辅助工具

转换与编码175

格式、文件、压缩包、单位与编码转换工具

潜客测验发布与联系人交接
讣告、悼词与纪念卡的家属复核及交付
本地习惯记录的备份、恢复演练与分享前检查
长文改编 X 串文与 LinkedIn 帖子的事实复核与交付
Markdown 互动课件审校、离线演示与交付

© 2026 Elysia Tools. 保留所有权利。

文档价格关于我们服务条款隐私政策Cookie 政策联系我们

Validation

AWS ARN校验器

验证Amazon资源名称(ARN)的格式

Media

批量图片重命名

使用模式、文本替换、编号和大小写转换批量重命名图片文件,返回重命名后的 ZIP 文件下载

Media

XMP元数据提取器

从图像中提取和查看XMP元数据,包括Adobe Photoshop、Lightroom和Creative Cloud信息

AI Tools

NSFW图片内容检测器

基于 NSFWJS 模型的 AI 内容审核工具,标记上传图片是否含不适宜内容。面向内容审核与合规自查场景,仅分析您上传的文件,不是图片搜索或发现工具。支持GIF/Animated WebP/APNG和JPEG/PNG/WEBP。

AI Tools

AI 年龄与性别估计

对检测到的人脸进行年龄与性别预测

AI Tools

AI 人脸对齐裁剪

检测并对齐人脸,裁剪成独立图片(多张打包 ZIP)

AI Tools

AI 人脸特征向量

为每张检测到的人脸生成 128 维特征向量

AI Tools

AI 人脸检测

检测图片中的人脸并返回边框坐标

云服务

AWS EventBridge 示例

AWS EventBridge 示例,包括事件总线、规则、目标、模式注册表、自定义事件和跨账户事件路由,适用于无服务器事件驱动架构

文档文件

PDF示例

2026-02-01 到 2026-02-10 工具生成的PDF示例

媒体

相机 RAW 照片样本

可下载的相机 RAW 样本文件集合,涵盖 Sony ARW/SRF/SR2、Canon CR2/CR3/CRW、Adobe DNG、Nikon NEF/NRW、Fuji RAF、Olympus ORF、Panasonic RW2、Leica RAW/DNG 以及一个通用 RAW 二进制文件,适合测试上传与转换流程。

监控

Grafana 高级应用示例

全面的 Grafana 示例,涵盖高级仪表板设计、告警配置、数据源集成和插件开发

媒体

SVG示例

可缩放矢量图形(SVG)示例,展示各种SVG功能和技术

Web 技术

WebRTC 实时通信示例

全面的 WebRTC 示例,用于点对点音视频通信、数据通道、屏幕共享和信令服务器实现