Running a InfluxDB Time-Series database on a AWS ECS cluster

Picking a TSDB

Running InfluxDB on AWS ECS

Persistant Storage

myPrivateEfsSg:
Type: AWS::EC2::SecurityGroup
Properties:
VpcId: !ImportValue myVpcID
GroupName: EfsPrivateSecurityGroup
GroupDescription: Security group for EFS mount
SecurityGroupIngress:
— IpProtocol: tcp
FromPort: 2049
ToPort: 2049
CidrIp: !ImportValue myPrivateSubnet1CidrBlock
— IpProtocol: tcp
FromPort: 2049
ToPort: 2049
CidrIp: !ImportValue myPrivateSubnet2CidrBlock
SecurityGroupEgress:
— IpProtocol: “-1”
CidrIp: “0.0.0.0/0”
ElasticFileSystemRetain:
Type: AWS::EFS::FileSystem
Condition: Retain
DeletionPolicy: Retain
Properties:
Encrypted: !FindInMap [ EncrpytionBoolean, !Ref EncryptionState, Boolean ]
KmsKeyId:
!If [ UseAWS-ManagedCMK, !Ref 'AWS::NoValue', !Ref Cmk ]
FileSystemTags:
- Key: Name
Value: !Ref 'AWS::StackName'
PerformanceMode: !Ref PerformanceMode
ElasticFileSystemDelete:
Type: AWS::EFS::FileSystem
Condition: Delete
DeletionPolicy: Delete
Properties:
Encrypted: !FindInMap [ EncrpytionBoolean, !Ref EncryptionState, Boolean ]
KmsKeyId:
!If [ UseAWS-ManagedCMK, !Ref 'AWS::NoValue', !Ref Cmk ]
FileSystemTags:
- Key: Name
Value: !Ref 'AWS::StackName'
PerformanceMode: !Ref PerformanceMode
ElasticFileSystemMountTarget0Retain:
Condition: Retain
DeletionPolicy: Retain
Type: AWS::EFS::MountTarget
Properties:
FileSystemId: !Ref ElasticFileSystemRetain
SecurityGroups:
- !Ref myPrivateEfsSg
SubnetId: !Ref Subnet1
ElasticFileSystemMountTarget0Delete:
Condition: Delete
DeletionPolicy: Delete
Type: AWS::EFS::MountTarget
Properties:
FileSystemId: !Ref ElasticFileSystemDelete
SecurityGroups:
- !Ref myPrivateEfsSg
SubnetId: !Ref Subnet1
ElasticFileSystemMountTarget1Retain:
Condition: Retain
DeletionPolicy: Retain
Type: AWS::EFS::MountTarget
Properties:
FileSystemId: !Ref ElasticFileSystemRetain
SecurityGroups:
- !Ref myPrivateEfsSg
SubnetId: !Ref Subnet2
ElasticFileSystemMountTarget1Delete:
Condition : Delete
DeletionPolicy: Delete
Type: AWS::EFS::MountTarget
Properties:
FileSystemId: !Ref ElasticFileSystemDelete
SecurityGroups:
- !Ref myPrivateEfsSg
SubnetId: !Ref Subnet2
Outputs:
ElasticFileSystem:
Value: !If [ Delete, !Ref ElasticFileSystemDelete, !Ref ElasticFileSystemRetain ]
Export:
Name: !Join ['', [ !Ref ExportName, 'ElasticFileSystem' ]]
ElasticFileSystemDnsName:
Description: DNS name for the Amazon EFS file system.
Value: !Join [ '.', [ !If [ Delete, !Ref ElasticFileSystemDelete, !Ref ElasticFileSystemRetain ], 'efs', !Ref 'AWS::Region', 'amazonaws', 'com' ] ]
Export:
Name: !Join ['', [ !Ref ExportName, 'ElasticFileSystemDnsName' ]]
ElasticFileSystemMountCommand:
Description: Mount command for mounting the Amazon EFS file system.
Value: !Join [ '', [ 'sudo mount -t nfs4 -o nfsvers=4.1,rsize=1048576,wsize=1048576,hard,timeo=600,retrans=2 ', !Join [ '.', [ !If [ Delete, !Ref ElasticFileSystemDelete, !Ref ElasticFileSystemRetain ], 'efs', !Ref 'AWS::Region', 'amazonaws', 'com:/' ] ] ] ]
Export:
Name: !Join ['', [ !Ref ExportName, 'ElasticFileSystemMountCommand' ]]

Brining up two InfluxDB containers

myInfluxDB1:
Type: AWS::CloudFormation::Stack
Properties:
Parameters:
mySubnet: !ImportValue myPrivateSubnet1
myLogGroup: "/ecs/influxdb1-ecs"
myServiceName: "influxdb1"
myContainerName: influxdb1-container
mySecurityGroup: !ImportValue PrivateClusterSG
TemplateURL: !Sub 'https://s3-${AWS::Region}.amazonaws.com/influxdb-ecs-nested-stack-${AWS::Region}-${AWS::AccountId}/stack-templates/influxdb-ecs-cf-template.yml'
TimeoutInMinutes: 20
myInfluxDB2:
Type: AWS::CloudFormation::Stack
Properties:
Parameters:
mySubnet: !ImportValue myPrivateSubnet2
myLogGroup: "/ecs/influxdb2-ecs"
myServiceName: "influxdb2"
myContainerName: influxdb2-container
mySecurityGroup: !ImportValue PrivateClusterSG
TemplateURL: !Sub 'https://s3-${AWS::Region}.amazonaws.com/influxdb-ecs-nested-stack-${AWS::Region}-${AWS::AccountId}/stack-templates/influxdb-ecs-cf-template.yml'
TimeoutInMinutes: 20
LogGroup:
Type: AWS::Logs::LogGroup
Properties:
LogGroupName: !Ref myLogGroup
TaskDefinition:
Type: AWS::ECS::TaskDefinition
Properties:
Family: influxdb-ecs
RequiresCompatibilities:
— EC2
NetworkMode: awsvpc
ExecutionRoleArn: !Sub arn:aws:iam::${AWS::AccountId}:role/ecsTaskExecutionRole
Memory: 512
Cpu: 256
ContainerDefinitions:
- Name: !Ref myContainerName
Image: influxdb:latest
Memory: 512
Cpu: 256
MountPoints:
— ContainerPath: /var/lib/influxdb
SourceVolume: influxdb_data_volume
PortMappings:
-
ContainerPort: 8086
HostPort: 8086
-
ContainerPort: 8083
HostPort: 8083
LogConfiguration:
LogDriver: awslogs
Options:
awslogs-group: !Ref myLogGroup
awslogs-region: !Ref AWS::Region
awslogs-stream-prefix: !Ref myServiceName
Volumes:
-
Host:
SourcePath: !Sub
— /efs/${ServiceName}
— {ServiceName: !Ref myServiceName}
Name: “influxdb_data_volume”

ServiceDefinition:
Type: AWS::ECS::Service
Properties:
LaunchType: EC2
TaskDefinition: !Ref TaskDefinition
Cluster: Hydro-Cluster-Private
ServiceName: !Ref myServiceName
ServiceRegistries:
— RegistryArn: !GetAtt InfluxDbServiceDiscovery.Arn
DesiredCount: 1
NetworkConfiguration:
AwsvpcConfiguration:
AssignPublicIp: DISABLED
SecurityGroups: [!Ref mySecurityGroup]
Subnets: [!Ref mySubnet]
InfluxDbServiceDiscovery:
Type: AWS::ServiceDiscovery::Service
Properties:
Name: !Ref myServiceName
DnsConfig:
DnsRecords: [{Type: A, TTL: “10”}]
NamespaceId: !ImportValue myPrivateServiceDiscoveryNamespace
HealthCheckCustomConfig:
FailureThreshold: 1
Outputs: MyInfluxDbServiceName:
Description: The Service discovery name
Value: !Join
— ‘.’
— — !GetAtt InfluxDbServiceDiscovery.Name
— !ImportValue myPrivateNamespace

--

--

--

Product Manager | Technology Consultant | Interim CTO | Entrepreneur | Investor

Love podcasts or audiobooks? Learn on the go with our new app.

Recommended from Medium

Dual booting Kali/Ubuntu on a gaming laptop!

How Software Developers Can Start Building Passive Income Streams Today

Networking with Swift + Combine.

AWS Certified Solutions Architect: Full Study List

How to install a maven on a windows system?

Comparison between Selection Sort and Insertion Sort

Laravel 8.x Login with Facebook Google Twitter and Github

Integrate Azure Key Vault with AKS — Using “FlexVolume” (Part 2/3)

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Stijn Vanorbeek

Stijn Vanorbeek

Product Manager | Technology Consultant | Interim CTO | Entrepreneur | Investor

More from Medium

SAP Netweaver Monitoring using Amazon CloudWatch

Deep Dive to Train, Deploy and Evaluate a Model in Amazon SageMaker

AWS DynamoDB — Primary Key and Secondary Index

Configuring Cache Control for S3 Hosted Resources