Ingest ADS-B feeds into Kimo

This is the hands-on companion to the whitepaper Airspace Awareness from Open Data. It assumes you know what ADS-B and Mode S are and want a pipeline you can trust. Everything here is civil, cooperative surveillance data; sample rows are simulated.

§01Which ADS-B source should you connect?

There are three realistic sources, and many teams end up with two of them. The right choice depends on the region you care about, whether you need navigation-quality fields, and what your license allows.

The OpenSky state vector exposes 18 fields per aircraft, including icao24, callsign, positions, barometric and geometric altitude, velocity, track, vertical rate, squawk, the special-purpose indicator, position_source (0 ADS-B, 1 ASTERIX, 2 MLAT, 3 FLARM) and, with extended=1, an aircraft category¹Source 1 · OpenSky NetworkOpenSky REST API documentationopenskynetwork.github.io. It does not carry NIC or NACp, so if you plan to build a GNSS interference layer you need either your own decoded messages or a historical source with raw operational-status messages. Published research did exactly that, merging OpenSky state vectors with NACp decoded from operational-status messages on icao24 and timestamp⁶Source 6 · Figuet, Waltert, Felux, Olive — Engineering Proceedings 28(1), 12, 2022GNSS Jamming and Its Effect on Air Traffic in Eastern Europedigitalcollection.zhaw.ch.

§02How many API credits will polling cost?

OpenSky meters /states/all by bounding-box area: a box of at most 25 square degrees costs 1 credit per request, rising to 4 credits for boxes over 400 square degrees or global queries. Daily quotas are 400 credits anonymous, 4,000 for a standard user and 8,000 for an active feeder; licensed users get 14,400 credits per hour¹Source 1 · OpenSky NetworkOpenSky REST API documentationopenskynetwork.github.io. Anonymous requests return only the latest vectors at 10-second resolution; authenticated requests have 5-second resolution and can look back up to one hour¹Source 1 · OpenSky NetworkOpenSky REST API documentationopenskynetwork.github.io.

The practical conclusion: research-tier polling of a region is a 30–60 second picture, which is fine for coverage maps, daily interference layers and retrospective analysis, and not fine for real-time alerting. If you need seconds-level latency, ingest from your own receivers (OpenSky's /states/own endpoint for your own sensors costs no credits¹Source 1 · OpenSky NetworkOpenSky REST API documentationopenskynetwork.github.io) or from a licensed feed.

§03Step 1: connect the source in Kimo

1. 01
   Step 1:

   Create the connector

   In Connectors, choose OpenSky Network or ADS-B Air Traffic. For own receivers streaming to a topic, choose Kafka.

2. 02
   Step 2:

   Set credentials

   OpenSky uses the OAuth2 client-credentials flow only; create an API client on your OpenSky account page and paste the client ID and secret. Tokens expire after 30 minutes and Kimo refreshes them automatically¹Source 1 · OpenSky NetworkOpenSky REST API documentationopenskynetwork.github.io.

3. 03
   Step 3:

   Declare the area and cadence

   Enter one or more bounding boxes (lamin, lomin, lamax, lomax) and a polling interval that fits your credit budget.

4. 04
   Step 4:

   Declare license basis and retention

   Record the license tier and set raw retention (we suggest 30 days for raw rows; aggregates live longer).

5. 05
   Step 5:

   Choose where data lives

   Pick Cloud mode to sync into Kimo, or Bridge mode so rows stay in your own database and Kimo queries through Kimo Bridge.

Before automating, test the call by hand. This is the documented token exchange and a bounding-box query¹Source 1 · OpenSky NetworkOpenSky REST API documentationopenskynetwork.github.io:

1234567891011121314export TOKEN=$(curl -s -X POST \
  "https://auth.opensky-network.org/auth/realms/opensky-network/protocol/openid-connect/token" \
  -H "Content-Type: application/x-www-form-urlencoded" \
  -d "grant_type=client_credentials" \
  -d "client_id=$CLIENT_ID" -d "client_secret=$CLIENT_SECRET" | jq -r .access_token)

curl -s -H "Authorization: Bearer $TOKEN" \
  "https://opensky-network.org/api/states/all?lamin=45.8389&lomin=5.9962&lamax=47.8229&lomax=10.5226&extended=1" \
  | jq '.time, (.states | length)'

# Check your remaining credits in the response headers
curl -s -D - -o /dev/null -H "Authorization: Bearer $TOKEN" \
  "https://opensky-network.org/api/states/all?lamin=45.8389&lomin=5.9962&lamax=47.8229&lomax=10.5226" \
  | grep -i x-rate-limit

For your own receivers, the bridge is the cleanest option when the decoder writes to a local Postgres or ClickHouse: install it with the Docker guide, add the database as a source, and nothing leaves your network except query results.

§04Step 2: land raw rows unchanged

Keep a raw table that stores exactly what the source returned, plus ingestion metadata: the response time, the request bounding box, the connector ID and ingested_at. Never fix data in the raw layer. When a decoding question comes up three months later ("why did this aircraft jump 40 km?"), the raw row is your evidence. OpenSky returns each state as a positional array, so the raw table can simply hold the JSON array; for own receivers, keep the raw hex frame alongside the decoded fields, as the OpenSky network itself does with its stored metadata³Source 3 · Schäfer et al. — ACM/IEEE IPSN, 2014Bringing Up OpenSky: A Large-scale ADS-B Sensor Network for Researchcs.ox.ac.uk.

Should raw aircraft data live in Kimo’s cloud or stay on your servers?

Raw state vectors are bulky, and for some teams they are sensitive because of what they reveal about local traffic. You can choose per source. In Cloud mode, Kimo syncs the raw and normalized tables into its managed storage, which is the fastest option for history and ad hoc exploration. In Bridge mode, the tables stay in your own database; Kimo Bridge opens an outbound-only, mutually authenticated tunnel and Kimo pushes queries down through it, storing nothing on its side unless you enable short-lived result caches. A common hybrid keeps raw vectors on your servers through the bridge and lets only hourly aggregates (coverage and quality cells) sync to the cloud for fast maps. The models in this guide are identical in both modes.

§05Step 3: normalize into a state-vector schema

Every downstream model reads from one typed table. The schema below is what the Airspace watch template ships with. Units are SI to match OpenSky (meters, m/s); convert to feet and knots only in the presentation layer.

Two columns deserve explanation. df matters because Downlink Format 18 is used by non-transponder ADS-B equipment and TIS-B ground rebroadcasts⁴Source 4 · Junzi Sun, TU Delft (mode-s.org)The 1090 Megahertz Riddle (2nd ed.) — ADS-B basicsmode-s.org; a TIS-B target is a ground system relaying radar-derived data and can duplicate a target you already see directly. nacp_age_s matters because NACp arrives in the less frequent operational-status message, so the value attached to a position is the last one received, and you should know how old it is. NIC and NACp are defined in US rules as the integrity containment radius and position accuracy of the reported position⁵Source 5 · Cornell Law School Legal Information Institute14 CFR § 91.227 — ADS-B Out equipment performance requirementslaw.cornell.edu; keep them even if you do not use them yet.

1234567891011121314151617181920212223insert into state_vectors
select
  to_timestamp(s[4]::bigint)                                as obs_time,
  lower(s[1]::text)                                         as icao24,
  nullif(trim(s[2]::text), '')                              as callsign,
  s[7]::double                                              as lat,
  s[6]::double                                              as lon,
  s[8]::double                                              as baro_alt_m,
  s[14]::double                                             as geo_alt_m,
  s[10]::double                                             as gs_ms,
  s[11]::double                                             as track_deg,
  s[12]::double                                             as vrate_ms,
  s[9]::boolean                                             as on_ground,
  s[15]::text                                               as squawk,
  s[17]::smallint                                           as position_source,
  null::smallint                                            as df,
  null::smallint                                            as nic,
  null::smallint                                            as nacp,
  null::int                                                 as nacp_age_s,
  1                                                         as receiver_count,
  r.connector_id                                            as feed
from raw_opensky_states r, unnest(r.states) as t(s)
where s[4] is not null;   -- skip vectors without a position timestamp

§06Step 4: deduplicate and check plausibility

Overlapping bounding boxes, multiple feeds and multiple receivers all produce duplicates. Deduplicate on address, observation time and rounded position, keeping the row heard by the most receivers. Then run a cheap kinematic check: if the distance between consecutive fixes implies a speed no civil aircraft can fly, flag the row. Most failures are decoding artifacts or position-encoding glitches, but some are worth an analyst's eye, and ADS-B has no authentication to rule anything out.

1234567891011121314151617181920212223242526272829303132333435create or replace view state_vectors_clean as
with ranked as (
  select *,
    row_number() over (
      partition by icao24, obs_time, round(lat, 4), round(lon, 4)
      order by receiver_count desc, feed
    ) as rn
  from state_vectors
  where lat is not null and lon is not null
),
dedup as (select * from ranked where rn = 1),
stepped as (
  select *,
    lag(obs_time) over w as prev_time,
    lag(lat) over w      as prev_lat,
    lag(lon) over w      as prev_lon
  from dedup
  window w as (partition by icao24 order by obs_time)
)
select *,
  -- haversine distance in meters
  2 * 6371000 * asin(sqrt(
      pow(sin(radians(lat - prev_lat) / 2), 2) +
      cos(radians(prev_lat)) * cos(radians(lat)) *
      pow(sin(radians(lon - prev_lon) / 2), 2)
  )) / nullif(extract(epoch from obs_time - prev_time), 0) as implied_ms,
  coalesce(
    2 * 6371000 * asin(sqrt(
      pow(sin(radians(lat - prev_lat) / 2), 2) +
      cos(radians(prev_lat)) * cos(radians(lat)) *
      pow(sin(radians(lon - prev_lon) / 2), 2)
    )) / nullif(extract(epoch from obs_time - prev_time), 0) > 350,
    false
  ) as implausible_jump   -- 350 m/s is well above civil cruise speeds
from stepped;

§07Step 5: segment flights with a versioned rule

A "flight" is a modeling decision. The OpenSky founders split messages into flights when an aircraft was silent for ten minutes, noting that a longer threshold wrongly merges quick turnarounds and a shorter one wrongly splits flights that leave and re-enter coverage³Source 3 · Schäfer et al. — ACM/IEEE IPSN, 2014Bringing Up OpenSky: A Large-scale ADS-B Sensor Network for Researchcs.ox.ac.uk. Use the same idea, but make the threshold a parameter, record the rule version on each flight, and add a ground-state boundary so a landing always ends a flight.

12345678910111213141516171819202122232425262728create or replace table flights as
with marked as (
  select *,
    case
      when prev_time is null then 1
      when obs_time - prev_time > interval '10 minutes' then 1
      when lag(on_ground) over w = true and on_ground = false then 1
      else 0
    end as new_flight
  from state_vectors_clean
  window w as (partition by icao24 order by obs_time)
),
numbered as (
  select *, sum(new_flight) over (partition by icao24 order by obs_time) as seg
  from marked
)
select
  icao24 || '-' || strftime(min(obs_time), '%Y%m%d%H%M') as flight_id,
  icao24,
  any_value(callsign)          as callsign,
  min(obs_time)                as first_seen,
  max(obs_time)                as last_seen,
  count(*)                     as positions,
  max(baro_alt_m)              as max_baro_alt_m,
  sum(case when implausible_jump then 1 else 0 end) as implausible_jumps,
  'v2-gap10m-ground'           as segmentation_rule
from numbered
group by icao24, seg;

§08Step 6: build a coverage model

Coverage is the denominator for everything else. Without it, an empty patch of map is ambiguous and trends are meaningless. Compute it from your own clean positions: per hexagonal cell, altitude band and hour, count distinct aircraft and positions. Cells with no recent observations are "unknown", not "empty".

1234567891011121314create or replace table coverage_cells as
select
  h3_latlng_to_cell(lat, lon, 5)                   as h3_cell,
  date_trunc('hour', obs_time)                     as hour,
  case
    when baro_alt_m < 3000  then 'low'
    when baro_alt_m < 7500  then 'mid'
    else 'high'
  end                                              as alt_band,
  count(distinct icao24)                           as aircraft,
  count(*)                                         as positions
from state_vectors_clean
where not on_ground
group by all;

Resolution 5 cells average roughly 250 km², which works for regional pictures. Use a coarser resolution if your network is sparse; the interference guide explains how to choose based on aircraft per cell per day.

§09Step 7: declare the models in Kimo

Wrap the tables in Kimo models so measures are defined once and reused by the Airspace map, alert rules and Ask Kimo. The YAML lives with your other models and is versioned.

123456789101112131415161718192021222324252627282930313233models:
  - name: state_vectors
    source: state_vectors_clean
    primary_key: [icao24, obs_time]
    time_dimension: obs_time
    dimensions:
      icao24: { type: string, description: "24-bit address; not an identity" }
      callsign: { type: string }
      squawk: { type: string }
      position_source:
        type: enum
        values: { 0: ADS-B, 1: ASTERIX, 2: MLAT, 3: FLARM }
      alt_band: { sql: "case when baro_alt_m < 3000 then 'low' when baro_alt_m < 7500 then 'mid' else 'high' end" }
    measures:
      aircraft: { sql: count(distinct icao24) }
      positions: { sql: count(*) }
      implausible_share: { sql: avg(case when implausible_jump then 1.0 else 0 end), format: percent }
    retention: 30d

  - name: flights
    source: flights
    primary_key: [flight_id]
    time_dimension: first_seen
    measures:
      flights: { sql: count(*) }
      median_duration_min: { sql: "median(extract(epoch from last_seen - first_seen) / 60)" }

  - name: coverage_cells
    source: coverage_cells
    time_dimension: hour
    measures:
      covered_cells: { sql: count(distinct h3_cell) }
      aircraft_per_cell: { sql: avg(aircraft) }

§10Step 8: validate before you trust it

§11Troubleshooting common ingestion problems

Once ingestion is stable, continue with GNSS interference detection and alerting rules, or start from the Airspace watch template, which contains these models pre-wired to simulated data.