Posts about software development

Kibana proxy for AWS ECS (Fargate) edit

23 January 2019

One of the most annoying parts of setting up Elastic Search on AWS is the way access is configured for that service. From what i've gathered you have one of two options to protect it:

  1. Make sure it's running internally only (i.e. not internet facing and only available to the other services on the same VPC)
  2. Completely open to the internet and eventually relying (if you can make it work) on the Cognito authentication service

Typically you would, as we did, go for the path of least resistance and greater security, which is to make it accessible only to the internal VPC. But that means you now cannot access Kibana to interact with ES.

The way we went around this at Drover was to setup a container running an Nginx proxy that provides basic auth protected access to kibana (using SSL). Here's how we did it:

1. The container

The docker container definition (Dockerfile)

FROM nginx:1.15

RUN apt-get update -qq && apt-get install -y --allow-unauthenticated openssh-server nano curl

# PROXY PORT
EXPOSE 80:80
# SSH
EXPOSE 22:22

# SSH access
RUN mkdir /root/.ssh
RUN touch /root/.ssh/authorized_keys
RUN chmod 700 /root/.ssh
RUN chmod 644 /root/.ssh/authorized_keys
RUN mkdir /var/run/sshd

# SSH public keys
RUN echo "ssh-rsa <public_key> void@mypc" >> /root/.ssh/authorized_keys

RUN mkdir /auth
RUN chmod 701 /auth
COPY /kibana.htpasswd /auth
RUN chmod 644 /auth/kibana.htpasswd
# TODO: Rest of envs
RUN chown -R www-data /auth

RUN rm /etc/nginx/conf.d/default.conf
COPY /nginx.conf /etc/nginx/conf.d/kibanas.conf
COPY /start.sh /
RUN chmod +x /start.sh

CMD /start.sh

Notes:

  • Nginx listens on port 80, but we also expose port 22 so we can ssh into the container for troubleshooting
  • The appropriate authorized keys are set into the container for passwordless login
  • The password file is copied into the appropriate location so it can be later read by nginx
  • The nginx config file is also copied to the correct location
  • Docker will run the start script when booting

The start script (start.sh)

#!/bin/bash

service ssh start
nginx

The nginx config file (nginx.conf)

server {
  listen 80;
  server_name kibanaz.myapp.com;

  auth_basic           "Restricted";
  auth_basic_user_file /auth/kibana.htpasswd;

  # redirect /
  location = / {
   rewrite ^ /_plugin/kibana/ redirect;
  }

  location / {
    proxy_http_version 1.1;
    proxy_set_header   Authorization ""; # Don't pass auth to kibana
    proxy_set_header   Upgrade $http_upgrade;
    proxy_set_header   Connection 'upgrade';
    proxy_set_header   Host $host;
    proxy_cache_bypass $http_upgrade;

    proxy_pass https://<your_url_to_es>/;
  }
}

Notes:

  • Make sure you have a DNS entry (either route53 or another) that points to this server name
  • The <your_url_to_es> should be the Elastic Search url you can find in the ES AWS service for the particular instance you want access to
  • This single proxy could potentially serve more than one Kibana/ES access, just add more server definitions

The password file (kibana.htpasswd)

<username_you_want>:<your_passwd_encrypted_password>

Notes:

2. Terraforming makes it easier

If you read one of my previous posts, you know that terraform is an excellent tool for setting up infrastructure. So that's what we use to setup the container in AWS ECS.

I will skip the cluster setup and instead focus on the task-definitions and services. Setting up resources using terraform is something that you can easily learn about in their documentation.

Task definition

data "aws_ecs_task_definition" "kibana" {
  task_definition = "${aws_ecs_task_definition.kibana.family}"
}

resource "aws_ecs_task_definition" "kibana" {
  family                   = "kibana"
  task_role_arn            = "arn:aws:iam::<your_amazon_account_id>:role/ecsTaskExecutionRole"
  execution_role_arn       = "arn:aws:iam::<your_amazon_account_id>:role/ecsTaskExecutionRole"
  network_mode             = "awsvpc"
  placement_constraints    = [],
  requires_compatibilities = ["FARGATE"]
  cpu                      = 256
  memory                   = 2048
  container_definitions    = <<DEFINITION
[
  {
    "name": "KibanaContainer",
    "image": "<your_amazon_account_id>.dkr.ecr.eu-west-2.amazonaws.com/kibana-proxy:latest",
    "cpu": 0,
    "portMappings": [
      {
        "containerPort": 80,
        "hostPort": 80,
        "protocol": "tcp"
      }
    ],
    "essential": true,
    "mountPoints": [],
    "volumesFrom": [],
    "logConfiguration": {
      "logDriver": "awslogs",
      "options": {
        "awslogs-group": "/ecs/KibanaProxy",
        "awslogs-region": "eu-west-2",
        "awslogs-stream-prefix": "ecs"
      }
    }
  }
]
DEFINITION
}

Notes:

  • This definition makes certain assumptions, as follows
  • There is already an initial empty task definition named Kibana in your fargate account
  • <your_amazon_account_id> should be filled with whatever is your account id
  • You have built, tagged and pushed the previously defined docker image into amazon's container repository and the name and tag of the image are kibana-proxy:latest
  • This is deploying to the eu-west-2 region (London), change it to closer to home if you need

The service

resource "aws_security_group" "kibana_sg" {
  name        = "kibana_sg"
  description = "Kibana proxy security group"
  vpc_id      = "${var.vpc_id}"

  ingress {
    from_port        = 80
    to_port          = 80
    protocol         = "tcp"
    cidr_blocks      = ["0.0.0.0/0"]
    ipv6_cidr_blocks = ["::/0"]
  }

  ingress {
    from_port        = 22
    to_port          = 22
    protocol         = "tcp"
    cidr_blocks      = ["0.0.0.0/0"]
    ipv6_cidr_blocks = ["::/0"]
  }

  # allow all outbound traffic
  egress {
    from_port        = "0"
    to_port          = "0"
    protocol         = "-1"
    cidr_blocks      = ["0.0.0.0/0"]
    ipv6_cidr_blocks = ["::/0"]
  }

  tags {
    Name = "kibana_sg"
  }
}

resource "aws_ecs_service" "kibana" {
  name            = "kibana"
  cluster         = "${module.support_cluster.cluster_id}"
  task_definition = "${aws_ecs_task_definition.kibana.family}:${max("${aws_ecs_task_definition.kibana.revision}", "${data.aws_ecs_task_definition.kibana.revision}")}"
  desired_count   = "${var.replicas}"
  launch_type     = "FARGATE"

  load_balancer {
    target_group_arn  = "${aws_alb_target_group.kibana-target-group.arn}"
    container_port    = 80
    container_name    = "KibanaContainer"
  }

  network_configuration {
    subnets          = "${var.subnet_ids}"
    assign_public_ip = true
    security_groups  = ["${aws_security_group.kibana_sg.id}"]
  }
}

Notes:

  • vpc_id is a variable being passed into this module, you can hard code it with the VPC you have setup for your cluster
  • cluster_id is the name of the cluster that you create to house this service (for instance Kibana, or support)
  • The service will try to use the latest task definition available (hence the previous need to have already a task definition with that name, otherwise it will fail the first time it runs)
  • replicas is the number of containers you want to be running (used by the load balancer)
  • subnet_ids is an array of subnets that you create for your cluster (typically one public for the load balancers and one private for the containers) 
  • target_group_arn is setup from the load balancers definition (up next)

Load balancing

resource "aws_alb" "kibana-load-balancer" {
  name            = "kibana-lb"
  security_groups = ["<public_facing_security_group_id>"]
  subnets         = "${var.subnet_ids}"

  tags {
    Name = "kibana-load-balancer"
  }
}

resource "aws_alb_target_group" "kibana-target-group" {
  name        = "kibana-tg"
  port        = "80"
  protocol    = "HTTP"
  vpc_id      = "${var.vpc_id}"
  target_type = "ip"

  health_check {
    healthy_threshold   = "5"
    unhealthy_threshold = "2"
    interval            = "60"
    matcher             = "200,302,301,401"
    path                = "/_plugin/kibana/"
    port                = "traffic-port"
    protocol            = "HTTP"
    timeout             = "15"
  }

  tags {
    Name = "kibana-target-group"
  }
}

resource "aws_alb_listener" "kibana-lb-listener" {
  load_balancer_arn = "${aws_alb.kibana-load-balancer.arn}"
  port              = "443"
  protocol          = "HTTPS"
  ssl_policy        = "ELBSecurityPolicy-2016-08"
  certificate_arn   = "arn:aws:acm:eu-west-2:<your_amazon_account_id>:certificate/<certificate_key_for_lb_domain>"

  default_action {
    target_group_arn = "${aws_alb_target_group.kibana-target-group.arn}"
    type             = "forward"
  }
}

resource "aws_alb_listener" "kibana-lb-redirect-listener" {
  load_balancer_arn = "${aws_alb.kibana-load-balancer.arn}"
  port              = "80"
  protocol          = "HTTP"

  default_action {
    type             = "redirect"
    redirect {
      port        = "443"
      protocol    = "HTTPS"
      status_code = "HTTP_301"
    }
  }
}

Notes:

  • public_facing_security_group_id is the security group id for the load balancer, which should be open to the outside (internet)
  • vpc_id is the public vpc
  • certificate_key_for_lb_domain - basically the whole certificate_arn is supposed to be the arn of the certificate you generate for the domain. Say you have the domain myapp.com in route53. You should generate a certificate for the load balancer to use that contains the myapp.com domain so that SSL works properly. Make sure it's consistent with the nginx.conf definition as well

Bonus - auto DNS

resource "aws_route53_record" "dns-kibana" {
  zone_id = "<myapp.com_zone_id>"
  name    = "kibanaz.myapp.com"
  type    = "CNAME"
  ttl     = "300"
  records = ["${aws_alb.kibana-load-balancer.dns_name}"]
}

Notes:

  • This resource creates a DNS entry for the load balancer in the provided zone id. The load balancer module defined previously will feed into the record being setup here so that it all ties together.

The conclusion

When you have it all setup, you can just fire up terraform apply and it should take care of setting up everything for you, assuming you already pushed the docker image into your account's docker repository. Once everything is running kibanaz.myapp.com should direct you to the kibana interface, asking for the Basic Auth username/password you setup before allowing you inside.

There is some glue work to do to make this all work, but i know you can do it. Just reference the documentation on terraform and it should be super easy.

Happy Kibaning.

Terraform - Infrastructure as code edit

18 January 2019

We recently started using this outstanding devops tool called Terraform at Drover. Basically it allows us to describe how we want our environments to be and apply that "plan" automatically. It's a bit like Ansible and its playbooks concept but - IMO - does a better job at tracking what resources exist and in which state they are in.

Think about it like writing down what your environment looks like, in terms of pretty much everything (load balancing, dns, servers, networking, connectivity, service discovery, databases, caching, port management, number of replicas, etc) and then just asking the command to make it so. And it gets better: it can keep tabs on links between all these resources you setup, which means you can, for instance, automatically add a DNS entry to the load balancer you defined, or define the database url on your server based on what was created by terraform.

In practice, it dynamically adjusts to all changes you make to your resources once created some being braking changes that create new resources, others just updating the resources properties. It works outstandingly with Amazon AWS services, Google, Azure, a lots and lots of other cloud setups (you can even mix and match).

Definitely check it out, it will make your devops work so much easier.

JAVA vertical gradient background panel

24 January 2015

Recently i had to create a gradient background for one of the panels in our [POS application](http://inforviegas.pt/wp-content/uploads/2010/09/VOS.jpg), so i though i'd share the code.

You need to pass the top and bottom colors of the gradient and also make sure that any panels inside this one are transparent (i.e. setOpaque(false)), otherwise they will paint above it.

import java.awt.Color; import java.awt.GradientPaint; import java.awt.Graphics; import java.awt.Graphics2D; import java.awt.RenderingHints;

import javax.swing.JPanel;

public class VerticalGradientBackgroundPanel extends JPanel {

private Color topColor; private Color bottomColor;

public VerticalGradientBackgroundPanel(Color topColor, Color bottomColor) {

this.topColor = topColor; this.bottomColor = bottomColor; }

@Override protected void paintComponent(Graphics grphcs) {

super.paintComponent(grphcs);

Graphics2D g2d = (Graphics2D) grphcs;

g2d.setRenderingHint(RenderingHints.KEY_ANTIALIASING, RenderingHints.VALUE_ANTIALIAS_ON); GradientPaint gp = new GradientPaint(0, 0, topColor, 0, getHeight(), bottomColor); g2d.setPaint(gp); g2d.fillRect(0, 0, getWidth(), getHeight()); } }

Displaying posts 1 - 3 of 62 in total