Kafka Web Console - Zookeeper Register Form

Web console for Kafka messaging system

Running Kafka for a streaming collection service can feel somewhat opaque at times, this is why I was thrilled to find the Kafka Web Console project on Github yesterday.  This Scala application can be easily downloaded and installed with a couple steps.  An included web server can then be launched to serve it up quickly.  Here’s how to do all that.

For a quick intro to what the web console does, see my video first.  Instructions for getting started follow below, including the quick video.

Kafka Web Console Project – Download

The main repository for this project is available on Github (claudemamo/kafka-web-console).  However, I wanted the ability to add and remove Kafka topics so I use a forked repository that has a specific branch with those capabilities.  (These are planned to be added to the main project but have not been yet.)

Download the ZIP archive file and unzip.

Before doing anything further, we need another application to build and launch the console.

Download Play Framework

A program called Play with Activator is used to build and launch the web console.  It’s a Java app for launching Scala apps.

Download it here, unzip it and add it to the system path so you can execute the activator command that it provides.

Build/Launch

Now back to the Kafka web console code.  Enter the top level directory and execute the Play Activator start command (with one special option):

cd kafka-web-console-topic-add-remove
activator start -DapplyEvolutions.default=true

 [info] Loading project definition from /home/demo/src/tmp/kafka-web-console-topic-add-remove/project
 [info] Set current project to kafka-web-console (in build file:/home/demo/src/tmp/kafka-web-console-topic-add-remove/)

 (Starting server. Type Ctrl+D to exit logs, the server will remain in background)

 Play server process ID is 8528
 [info] play - database [default] connected at jdbc:h2:file:play
 [info] play - Starting application default Akka system.
 [info] play - Application started (Prod)
 [info] play - Listening for HTTP on /0:0:0:0:0:0:0:0:9000

The first time it runs will take some time to build the source and then launch the web server.

In the last line, above, you see it launches by default on port 9000.

Configuring the Console

Kafka Web Console - Zookeeper Register Form
Step 1 – Register your Zookeeper instance

Now you can launch the web console and start using the application.  Step 1 figure shows the basic form for your Zookeeper configuration details.  This is the only setup step required to get access to your Kafka brokers and topics.

The remained of the steps/figures are just showing the different screens.  See the video for watching it in action.

Kafka Web Console - Broker List Table
Step 2 – Brokers tab shows list of Kafka Brokers
Kafka Web Console - Topic List Table
Step 3 – Topics tab lists the Kafka topics. Note that this forked version provides the red “delete” button for each topic, as well as the additional “create topic” tab above.
Kafka Web Console - Topic Feed Table
Step 4 – Selecting a topic allows you to see an active topic stream.

 

Published by

Tyler Mitchell

Product and marketing leader, author and technology writer in NoSQL, big data, graph analytics, and geospatial. Follow me @1tylermitchell or get my book from http://locatepress.com/gpt

5 thoughts on “Web console for Kafka messaging system”

  1. Hi,

    I tried this but the server wouldn’t start. From the logs it has a warning:

    2015-03-27 21:25:27,312 - [WARN] - from play in main
    Your production database [default] needs evolutions!

    It also outputs some exceptions:

    CREATE TABLE groups (
    @6lip8i8m2: Database 'default' needs evolution!id LONG,

    name VARCHAR,
    at play.api.db.evolutions.EvolutionsPlugin$$anonfun$onStart$1$$anonfun$apply$1.apply$mcV$sp(Evolutions.scala:484)
    PRIMARY KEY (id)
    at play.api.db.evolutions.EvolutionsPlugin.withLock(Evolutions.scala:507)
    );
    at play.api.db.evolutions.EvolutionsPlugin$$anonfun$onStart$1.apply(Evolutions.scala:461)

    at play.api.db.evolutions.EvolutionsPlugin$$anonfun$onStart$1.apply(Evolutions.scala:459)
    CREATE TABLE status (
    at scala.collection.immutable.List.foreach(List.scala:318)
    id LONG,
    at play.api.db.evolutions.EvolutionsPlugin.onStart(Evolutions.scala:459)
    name VARCHAR,
    at play.api.Play$$anonfun$start$1$$anonfun$apply$mcV$sp$1.apply(Play.scala:88)
    PRIMARY KEY (id)
    at play.api.Play$$anonfun$start$1$$anonfun$apply$mcV$sp$1.apply(Play.scala:88)
    );
    at scala.collection.immutable.List.foreach(List.scala:318)

    at play.api.Play$$anonfun$start$1.apply$mcV$sp(Play.scala:88)

    Any ideas?

    Thanks

  2. I got that error at first, but then ran it with the applyEvolutions option and it cleared up:

    activator start -DapplyEvolutions.default=true

    Not sure about all those exceptions though, maybe this will help. Let me know! Thanks for reading.

  3. Hi Tyler,

    Thanks for getting back to me. I got it working in the end by putting the property before the start command:

    activator -DapplyEvolutions.default=true start

    So the ordering of the commands made a different to pick up the evolutions property. Hope this helps others.

Comments are closed.

WordPress Appliance - Powered by TurnKey Linux