EzDevInfo.com

Automating Amazon EBS snapshots anyone have a good script or solution for this on linux [closed]

I'd expect this to be fairly routine, but cannot find a simple approach for creating an managing EBS snapshots automatically.

Was hoping there'd be a shceduler in the AWS console.. alas not yet.

Would appreciate any suggestions as to how best to do this on from Ubuntu.

Thanks

Automatically mount an EBS volume upon starting an Amazon EC2 Linux instance

I have an EBS volume (e.g. /dev/sdf) that has been attached to an EC2 instance (which boots from a different EBS volume), and I have mounted the volume (through mount /dev/sdf /data). When I stop and start again the instance, the volume is still attached but no longer mounted, and I have to manually mount it again.

Is there a way to make the volume /dev/sdf automatically mounted to /data upon starting the instance?

What data is stored in Ephemeral Storage of Amazon EC2 instance?

I am trying to stop a Amazon EC2 instance and get the warning message

Warning: Please note that any data on the ephemeral storage of your instance will be lost when it is stopped.

My Question

What data is stored in ephemeral storage of an Amazon EC2 instance?

How to make a daily back up of my ec2 instance? [closed]

I have a community AMI based Linux EC2 instance in AWS. Now I want to take a daily back up of my instance, and upload that image in to S3.

Is that the correct way of doing the back up of my EC2 instance? Can anybody help me to point out the correct method for taking back up of my EC2 instance?

How do I increase the EBS volume size of a running instance?

I have a server running the recent Ubuntu AMIs from Canonical. The size of the EBS boot volume is 8GB. I know that I can resize EBS volumes by taking a snapshot, creating a new volume and expanding the partition on it. How can I increase the size of the volume while the machine is running? If this is not possible, what is the preferred method for increasing the boot volume size with minimal downtime?

How do the amazon web services work?

What information have you been able to gather regarding how do the amazon web services work?

• What hardware do they use
• What web server
• What Operating System
• What storage for AWS
• What virtualization software for EC2/EBS
• What software for they distributed firewall for EC2
• Physical location of their data centers.

I like their services very much and use them a ton at work... just out of curiosity. If you know/heard/read and want to tell, if you saw something online and want to provide a link, very appreciated.

EBS for storing databases vs. website files

I spent the day experimenting with AWS for the first time. I've got an EC2 instance running and I mounted an Elastic Block Store (EBS) to keep the MySQL databases.

Does it make sense to also put my web application files on the EBS, or should I just deploy them to the normal EC2 file system?

What effect does deleting EC2 snapshots have?

I'm not sure what effect deleting an EC2 snapshot has on the other ones. For example, if I snapshot an EBS volume 4 times and delete the oldest one, can I still do a full restore from the latest ones? In other words, is there any benefit in keeping old snapshots other than to save incremental changes?

Benefits of EBS vs. instance-store (and vice-versa)

I'm unclear as to what benefits I get from EBS vs. instance-store for my instances on Amazon EC2. If anything, it seems that EBS is way more useful (stop, start, persist + better speed) at relatively little difference in cost...? Also, is there any metric as to whether more people are using EBS now that it's available, considering it is still relatively new?

Can someone elaborate on these basic concepts of hosting via Amazon Web Services?

I read all over the Amazon documentations, but I'm still confused or should I say overwhelmed by the different terms.

I'm coming from a traditional web hosting environment where the only concept I understand is how much storage I have and how much bandwidth I'm allowed.

Here are what I understand so far about amazon and my questions about a lot of it.

EC2 - I assume are instances where I can set up my webserver (IIS) and run my .NET application? Or is it already setup for me?

EBS - For database? If not, where do I get the database server? Is the database server (Sql server 2005 or 2008) already installed? What is Snapshot Get Requests? and Snapshot Put Request?

S3 - What is this used for? I thought EBS is for storage, confused here. Why you need S3?

Elastic Load Balancing - I thought load balancing is just a way to alleviate the burdens on your web servers. how does it work with Amazon? What do they mean by "elastic" load balancing?

Data Transfer between region - What does that mean? and how do you control which region the data transfers to and from?

My requirements are the following

1. I need an IIS webserver to run my page
2. I need a database server
3. I need a location to store my files (can it be on the same "server" as #1)?
4. I need the database and file servers to always be recoverable. (I heard we need to store it into EBS to avoid loss of data?)
5. In case one web server is overloaded, performs slow, I need it to switch off to another server in the farm.
6. If traffic spikes, then I need a way to increase CPU, memory, etc. The ability to handle traffic nicely (which is the purpose of elastic I'm assuming)

In addition, I don't really understand the concept of "if you are not running your instance, shut it down or else it will be charged". I will run a website which ideally is to keep it up 24/7. In what scenarios do you conditionally have the instances up or down?

[Edit]

And how do you exactly monitor your usage? So you don't get surprised with a couple thousands of dollars? can you put a cap on it? I read stories where AWS users got surprised by a $300 bill or whatever when they didn't have much hosted on Amazon nor have any traffic intensive sites.

Sorry, I'm a complete newbie to "cloud" computing. Trying to catch up here.

Thanks a lot in advance

Backup solutions for AWS EC2 instances

I'm looking for a backup solution for Amazon EC2 instances. I found this: http://www.n2ws.com and I wanted to know if there were other ones.

Thank you

PS: It's possible to automatically backup RDS databases using Amazon solution but there isn't anything for EC2 instances... Is there?

Kubernetes on AWS cloud provider

I installed CentOS Atomic Host as operating system for kubernetes on AWS.

Everything works fine, but it seems I missed something.

I did not configure cloud provider and can not find any documentation on that.

In this question I want to know: 1. What features cloud provider gives to kubernetes? 2. How to configure AWS cloud provider?

UPD 1: external load balancer does not work; I have not tested awsElasticBlockStore yet, but I also suspect it does not work.

UPD 2: Service details:

$ kubectl get svc nginx-service-aws-lb -o yaml
apiVersion: v1
kind: Service
metadata:
  creationTimestamp: 2016-01-02T09:51:40Z
  name: nginx-service-aws-lb
  namespace: default
  resourceVersion: "74153"
  selfLink: /api/v1/namespaces/default/services/nginx-service-aws-lb
  uid: 6c28b718-b136-11e5-9bda-06c2feb29b0d
spec:
  clusterIP: 10.254.172.185
  ports:
  - name: http-proxy-protocol
    nodePort: 31385
    port: 8080
    protocol: TCP
    targetPort: 8080
  - name: https-proxy-protocol
    nodePort: 31370
    port: 8443
    protocol: TCP
    targetPort: 8443
  selector:
    app: nginx
  sessionAffinity: None
  type: LoadBalancer
status:
  loadBalancer: {}

How to scale MongoDB?

I know that MongoDB can scale vertically. What about if I am running out of disk?

I am currently using EC2 with EBS. As you know, I have to assign EBS for a fixed size.

What if the MongoDB growth bigger than the EBS size? Do I have to create a larger EBS and Copy & Paste the files?

Or shall we start more MongoDB instance and each connect to different EBS disk? In such case, I could connect to a different instance for different databases.

How to 'warm up' EC2/EBS storage?

I have seen advice to 'warm up' EC2 to overcome a first write penalty:-

Warm up data partitions - There is one drawback to using disk IO in EC2: a “first write” performance hit when initially writing to new partitions. To avoid this penalty, you can “warm up” the partition by executing a sort of throw-away command that accesses it. For example, you can use the Linux dd command to write to the disk. While the penalty still occurs and cannot be avoided, at least the first write to your databases will not suffer the effects.

Source: http://answers.oreilly.com/topic/1345-getting-the-most-out-of-mysql-in-the-amazon-cloud/

...but I haven't found any further advice on best practice! Is this true of EBS storage? Can anyone recommend the 'dd' syntax that will perform this warmup and how to ensure that all blocks are 'warmed'?

Improving Data Transfer Rate on Amazon EC2

I've got a 1-Gig EBS Volume mounted to an EC2 instance, I am copying 600MB of binary data from a Local Hard Drive (via RDS Connection) and the copying process windows is showing 10 Hours remaining. Though I have a High Speed connection (100+Mbps)

Whatever the data volume, data transfer rate is 1 Min / MB (i.e 16Kbs / Sec)
I am hesitating between reading Moby Dick in front of my workstation or just taking a day-off.

Are there any reasonable options to speed up this transfer rate ?
(Ideally 512 Kbps/Sec at the minimum)

I am very open to ANY solution to shorten up the uploading/downloading time to/from and EC2 instance.

Thanks in advance.

EDIT :
I just stumbled upon [Amazon Export/Import Service][1] "AWS Import/Export accelerates transferring large amounts of data between the AWS cloud and portable storage devices that you mail to us"
By "mail to us", they literally mean you "materially" shipping your storage device to Amazon.
Don't say this is Stoneage, this is BRAND new TECHNOLOGY, Dude ! :-)

EDIT2 :
This sounded great: [Aspera for AWs][2] But unfortunately way too expensive;
Tailored for Fortune 500 with big needs and big cash.