Welcome to Ensign U!

Welcome to Ensign U!

Rotational Labs | Friday, Oct 20, 2023 |  Python Eventing ML

Like any new tool, Ensign takes a little time to learn. But take it from us (a bunch of seasoned data science experts) – it will open up a new world of data science applications for you. Check out this beginner-friendly introduction!

Ensign is a flexible database and data streaming engine built for data scientists by data scientists. We make it easy for data teams to build and deploy real-time data products and services that make an impact, no matter your use case.

To help you on your learning journey, we’ve established Ensign U — a curriculum designed to get your first project up and running.

First, the Pre-Reqs

If you’re new to real-time data streaming, pubsub and event-driven architecture, here are a few recommended resources:

Make sure you have Python installed. We recommend using Python >= 3.9.

Step 1. Start with a Use Case

Start with a use case in mind. This can be any type of streaming data project that you’re interested in and your primary goal is to bring meaning to the data.

Looking for more inspiration?

Step 2. Pick Your Streaming Dataset(s)

Where is your data coming from? A data source can be a database, a public data API, a running web application, a bunch of IoT sensors, a file system, system logs, or other upstream sources.

Check out our Data Playground for publicly available APIs that stream data.

Step 3. Sketch Out Your Data Flow Architecture

Most data scientists want to start by training the machine learning model, since that’s how we’re trained. But this can lead you to a trained model that can’t fit into an application without a lot of painful MLOps.

In our experience, data science projects are much easier to deploy if you map out your data flows first, and then start modeling.

Watch these short videos:

Step 4. Set Up Your Project Infrastructure

Here’s where you need to create an account and log into Ensign. These videos will help.

  • Create a project: A project is a collection of topics and will serve as your data repository for streaming events.
  • Name your topics: A topic is a log of events that contain information you’re interested in.
  • Generate API keys: Use your API keys to connect your data source to your topic.
  • Connect using your API keys: Establish a client connection to the Ensign server by passing in the ClientSecret and ClientID from your API keys, either via environment variables or a gitignored file inside the repo.

Step 5. Install PyEnsign

Install the pyensign sdk. We recommend using Python >= 3.9.

You can pip install pyensign with:

pip install pyensign

Or, if you have an older version of pyensign installed (especially before v0.11b0), make sure you are on the latest release by upgrading your pyensign version using:

pip install --upgrade pyensign

Check out the docs if needed!

Step 6. Run the Minimal PyEnsign Example

Check that your API keys and your pyensign installation are working. You can do this by running the minimal python example.

Step 7. Run the Earthquake Data Ingestion Example Code

Run the earthquake example to ingest data into your topic using your own API keys.

Step 8. Query Your Ingested Data

Once you get data into your topic, you can view the data by logging into your Ensign account and navigating to Projects > Project Name > Topic Name. You will see the details page for your topic, and you can run the sample Topic Query provided to view a few of the events you have successfully ingested.

You can also query your data using EnSQL — see an example of how to do that with PyEnsign and your earthquake data here.


Now you have data flowing into your topic. The world is your oyster!

Use this same process each time you have a new data science project, and your chances of getting it deployed will be that much greater!

Here are some common data science projects that can add value to an organization:

  • Working with sensitive data? Apply transformations that mask or anonymize data from your data source.
  • Create real-time machine learning models that learn on the go.
  • Publish streams to web apps, mobile apps, dashboards, an
  • Publish data from different topics into a new a new topic
  • Collect data from edge devices and sensors.

Photo by Pablo Dodda on Flickr Commons.

About This Post

Your end-to-end curriculum for getting started with Ensign, a database-meets-event-streaming platform for data science teams.

Written by:

Share this post:

Recent Rotations butterfly

View all

PubSub 101 - Creating Data Flows With Topics

If you struggle to build analytics and models that keep up with changing data, it might be because you haven’t yet learned to think about data in terms of topics. In this module, learn how by solving a real-world, real-time problem!

Nov 22, 2023

Building Real-Time Apps in Python with Ensign and Streamlit

Python may be the 2nd best language for everything, but it’s a favorite of data scientists worldwide, and delivers a world of functionality. Did you know you can even build customer-facing AI/ML apps with Python now? Learn how…

Nov 20, 2023

PubSub 101 - Using the PyEnsign SDK

The Python SDK is the most popular way to use Ensign. In this module you will write some Python code to publish data to your project.

Nov 17, 2023
Enter Your Email To Subscribe