技术文档

M5c step 3: navdata from MSFS for Zibo's FMS

此页为项目的英文技术文档原文。 docs/M5C_NAVDATA.md

Status 2026-11-07: design; the X-Plane folder in the XLua host (build 50); native loaders equal to Zibo's (build 51); the MSFS crawler, the files and the FMS reload (build 52); terminal procedures as CIFP files (build 53). To be checked in the sim. The appendix is the interface Zibo's FMS expects (tables, files, load order), with line numbers in B738.a_fms.lua.

Design#

Zibo's FMS sees navdata only through Lua tables it fills at flight_start (one synchronous call) from X-Plane's text files, and per airport from CIFP files when the pilot enters an origin / destination (appendix §1, §4). The provider keeps that interface and changes only where the data comes from.

  1. X-Plane's folder = \work\xp\ (done, build 50): Lua's io.open puts the relative paths Zibo uses (Custom Data/earth_nav.dat, Custom Data/CIFP/KSEA.dat, Output/FMS plans/…) under the module's work folder and makes the folders a written file needs (xlua_set_root, wasm/src/core/fms/xlua_glue.c).
  2. A navdata cache in X-Plane's published formats, made from MSFS's database: earth_nav.dat (1150), earth_fix.dat (1101), earth_awy.dat (1100), earth_hold.dat, Zibo's own airport / runway caches (B738X_apt.dat, B738X_rnw.dat, with the header detect_apt_dat accepts) and CIFP/<ICAO>.dat per airport. Our files, written from SimConnect facility data (SimConnect_RequestAllFacilities, RequestFacilityData: VOR, NDB, WAYPOINT + ROUTE, AIRPORT + RUNWAY / DEPARTURE / ARRIVAL / APPROACH and their legs, HOLDING_PATTERN); no X-Plane data. The file format is the contract: Zibo's own parsers read it, so the tables are Zibo's.
  3. Native loaders: Zibo's parsers go character by character (synthetic world-sized data: 15.5 s in native PUC Lua, so about a minute in MSFS, in one frame). read_rnw_dat, read_navdata2, read_apt_dat, read_awy_dat2 and create_idx_table are replaced after the script loads by C functions that build the same tables (rows, order, index buckets) from the same files. Checked by table equality against Zibo's Lua loaders on the same files (native test, tools/fms_spike/gen_nav.py data and hand-made edge cases). Done, build 51: wasm/src/core/fms/xlua_nav.c, installed in the script's namespace after it loads (xlua_set_native_nav, on by default; Fms::nativeNav). read_navdata2 falls back to Zibo's own when Global Airports' earth_nav.dat, Custom Data/user_nav.dat or custom ILS data are there (files only Zibo's handles). wasm/test/test_navdata.cpp: every table, index and global the loaders fill (29) equal by digest to Zibo's loaders' on the same files (20,000 fixes, edge cases: VOR-DME pairs, encoded localizer courses, LOC / LDA / IGS, GLS / LPV, CR line ends, short lines, double spaces, tabs, non-numbers, hyphenated airways, TL as FLnnn); 3.4 s with Zibo's loaders, 0.6 s with these (test build, both including the FMS load).
  4. When: the first flight with no cache (or a stale one) builds it in the background (progress in an L:var) and the FMS loads when it is ready; later flights load the cache at once. CIFP files are written on demand and prefetched for the airports in the flight plan; the first entry of an airport not fetched yet shows what Zibo shows without data (NOT IN DATA BASE) until it is there.
  5. AIRAC: MSFS does not name its cycle; the cycle in the header is the one current on the cache's date, and the cache is rebuilt when the date passes the cycle's end or MSFS's facility counts change.
  6. Magnetic variation: Zibo asks zibomod for the variation at a point (zibomod/lat_mag_var, zibomod/mag_var); the port answers with the aircraft's. A world magnetic model in C (WMM) answers per point.

Build 52: fms/navdb.cpp (records → X-Plane's formats, Zibo's caches with the header his detect_apt_dat accepts, runway ends sorted and measured as his create_apt_rnw_dat2), fms/nav_crawl.cpp (the whole world asked for a batch at a time: VOR / ILS, NDB, airport + runways, enroute waypoint + airways; facts in docs/MSFS_FACILITY_DATA.md), fms/nav_provider.cpp (cache of the current cycle, or crawl → files → FMS reload; L:vars in docs/OPTIONS.md), the gauge's SimConnect definitions and dispatch. Tests: the writer's files read by Zibo's loaders (and ours, equal), the crawler against a fake facility source, the provider's crawl → reload → cache hit. Known gaps: airway segments between two navaids (VOR–VOR) are not listed by a waypoint's routes; the route TYPE → high / low mapping is a guess; terminal procedures (CIFP) not yet.

Build 53, procedures: fms/nav_procs.cpp: an airport's facility data with its runways, departures, arrivals and approaches (one SimConnect definition, the items put together by their parent ids, else by the procedure / transition they follow) → Custom Data/CIFP/<ICAO>.dat in X-Plane's CIFP format (XP CIFP 1101/1102: ARINC 424 fields; trailing spaces trimmed, a blank field one space, the description code its 4 columns, RWY: designator 5 and localizer 4 columns): SID runway transitions (route type 1), common route (2; ALL when the procedure has no runway transition), enroute transitions (3); STAR enroute (1), common (2), runway (3); approach transitions (A) then the final and missed approach (the approach's type letter, the first missed leg marked M in the description's 3rd column); approach idents as I16L, R09-Y, R09RY. Units: altitudes metres → feet (5 digits; B: the upper limit first), distances metres → tenths of a NM, hold time in tenths of a minute (T010), courses tenths of a degree, vertical angle hundredths (-300), speed limits at or below. When: NavProvider asks for an airport when Zibo's FMS opens its CIFP file and it is not there (the XLua host reports read misses), and for the 3 airports within 25 nm of the aircraft once the navdata is ready; the CIFP files are removed when a new cycle is read. Tests: the file read by Zibo's own read_ref_data / read_des_data (SIDs by runway, SID transitions, STARs as ALL, the approach and its transition, runways, legs with names, altitudes, FAF, first missed leg, hold); the shipped module in fake_msfs_host.py (aircraft at KSEA: the procedures fetched and written).

Build 55, lists first and on demand (owner, build 54: the world lists came at once, 349,087 facilities; the details one by one would have taken hours, 4.4 % after minutes): the lists are written as the files at once (every waypoint, navaid, airport with its position) and the FMS reloaded; the VORs' / NDBs' details follow in the background, nearest first, and complete the files for the next load (lu737_navdata_complete.txt); airways (Zibo's find_awy wrapped: a miss is reported through lu737_call, the airway followed from the fix both ways), an airport's runways / transition levels / localizers (with its procedures, when Zibo opens its CIFP file) come on demand and are added to the FMS's tables at once (Lua helpers in the script's namespace: rows and first-character indexes as Zibo builds them; wasm/src/core/fms/fms.cpp kNavGlue) and appended to the cache files.

Order of work: (a) native loaders with equality tests; (b) the SimConnect crawler and the file writers; (c) CIFP from MSFS procedures; (d) magnetic variation; (e) airport elevation handshake (zibomod/find_icao_cmd).


Appendix: Zibo's navdata interface#

Source: plugins/xlua/scripts/B738.a_fms/B738.a_fms.lua (172,900 lines). All line numbers refer to that file unless another file is named. "word N" means the N-th whitespace token of a line, as Zibo splits it (runs of spaces collapse; a trailing CR is stripped from the last token). "field N" in CIFP means the N-th comma-separated token.

Port facts that matter (checked in this repo):

  • io.open is wrapped in the port (wasm/src/core/fms/xlua_glue.c:848 xlua_l_io_open): relative paths are put under \work\xp\ (wasm/src/core/fms/fms.cpp:176). zibomod/Aircraft_Path (= file_path2, the folder for the B738X_* caches) is set to workDir (fms.cpp:191-194). The port calls flight_start once after the scripts load (fms.cpp:207).
  • All navdata tables are "raw tables" in the XLua namespace: raw_table('earth_nav') etc. (plugins/xlua/init.lua namespace_write 376, get_raw_table_in_namespace 471). A native filler must store them as plain tables in the script namespace's values (the same thing earth_nav = {} does in Lua), not in _G. Every global read of earth_nav, idx_earth_nav and so on goes through the namespace_read metamethod (init.lua:413).
  • The Lua code uses only :read() (no argument = one line) and :close() on read handles (all file_navdata* handles). A virtual file object with just those two methods would be enough to feed the CIFP parser.

1. Load sequence and call graph#

1.1 At flight start: one synchronous call, all in one frame#

flight_start() (170237). Navdata is loaded only if plane_already_loaded == 0 (170609; global init 539, set to 1 at 170682, never reset — aircraft_unload 170230 does not reset it). So it loads once per Lua state, all inside one call (no timers, no frame slicing).

flight_start                              170237
  FILE_NAME_* = "B738X_apt.dat" / "B738X_rnw.dat" / "B738X_gate.dat" / "B738X_jetway.dat"   170430-170433
  FILE_NAME_CUST_DAT = "Custom Data/B738X_wptx.dat"                                        170434
  update_apt_dat = 0                                                                       170602
  if plane_already_loaded == 0:                                                            170609
    update_config()                                                                        170611
    detect_apt_dat()            8453  -> may call create_apt_rnw_dat2() 8630, sets update_apt_dat = 1   (8622-8625)
    earth_nav_num=0; earth_nav={}; ils_nav_num=0; ils_nav={}; gls_nav_num=0; gls_nav={}    170616-170621
    rnw_data_num=0; rnw_data={}                                                            170623-170624
    read_rnw_dat()              9491  (must run BEFORE check_navdata: read_navdata2 uses rnw_data)
    check_navdata()             7105  -> read_navdata2() 7332 -> find_airac_cycle2(airac_cycle) 8405 (called at 8028)
    wptx_n=0; wptx={}; load_cust_wpt()   17865  (Custom Data/B738X_wptx.dat, user waypoints, not navdata)
    apt_data_num=0; apt_data={}; read_apt_dat()   9394
    awy_data_num=0; awy_data={}
    if airac_navdata ~= "" then read_awy_dat2() end   9572   (170642: airways load ONLY if the earth_nav.dat
                                                               header had "cycle ")
    create_idx_table()          8177
    awy_path={}, awy_path_num=0, awy_temp_num2=0, awy_temp2={}
    B738_detect_fmod(); sfp_old = B738DR_sfp
    load_navdata = 0            170659  (set only; never read anywhere)
  plane_already_loaded = 1      170682

Table resets before the first load: top-level script code at 1982-1987 (apt_data, rnw_data), 2137-2138 (awy_data), 2667-2697 (earth_nav, ils_nav, gls_nav, idx_gls, idx_gls_icao, hold_nav, idx_hold, airac_navdata="", airac_navdata_active="", airac_cycle=""), 2932-2946 (cust_def_data=0, update_apt_dat=0, idx_earth_nav, idx_apt, idx_ils, idx_ils_icao, idx_rnw, idx_awy).

1.2 Gating globals, flags and datarefs#

Name Defined Meaning
plane_already_loaded 539 0 → load navdata in flight_start; set to 1 after.
cust_def_data 2932, set in check_navdata 0 = no navdata found (no navdata, no CIFP ever opened); 1 = Custom Data/; 2 = Resources/default data/. Every CIFP open branches on it (54 sites).
file_name_path, file_name_path_nav/_fix/_awy check_navdata Directory prefix ("Custom Data/" or "Resources/default data/").
update_apt_dat 2933, 170602, 8624 1 if detect_apt_dat rebuilt the B738X caches; forces Custom Data/user_nav.dat to be regenerated (7168).
custom_ils_status dataref laminar/b738/custom_navdata_status (4756) 0 off; 1 "regenerate user_nav.dat now"; 2 regenerated (7964-7968).
B738DR_cust_nav_data dataref laminar/B738/effects/cust_nav_data (4165) Config option "CUST NAVDATA" (43932-43938, cfg slot 105). ≠0 asks for the user_nav.dat merge (7125).
airac_cycle 2702 4 chars after "cycle " in the earth_nav.dat header (7619).
airac_navdata 2700 string.sub(build,3,8) .. airac_cycle (7619-7622), e.g. build 20231228, cycle 2401 → "2312282401". Gates read_awy_dat2 (170642). Shown on IDENT NAV DATA (108270-108272).
airac_navdata_active 2701 find_airac_cycle2 → "MMMddMMMdd/yy" (13 chars, e.g. "JAN25FEB21/24"). IDENT page; B738_fmc_msg (135260-135290) parses chars 6-8/9-10/12-13 for NAV DATA OUT OF DATE (vs os.date("!...")) and posts CORRUPTED NAVDATA if shorter than 13 chars.
msg_nav_data 1467, 164900, 165634 State of the two messages above.
load_navdata 540, 170659 Written only.
zibomod/Aircraft_Path → file_path2 6234 Folder for the B738X_* caches.
zibomod/mag_var family 3780-3783 Position mag var service (see §5).
zibomod/find_icao_cmd, laminar/B738/find_icao, laminar/B738/find_icao_height 6581-6585 Airport elevation service (see §5).

1.3 Files opened (exact strings; relative to X-Plane root = \work\xp\ in the port)#

check_navdata (7105):

  1. Custom Data/user_nav.dat (read 2 lines, 7126-7144).
  2. Custom Data/earth_nav.dat, Custom Data/earth_fix.dat, Custom Data/earth_awy.dat (open test; if B738DR_cust_nav_data ~= 0, reads the first 2 lines of earth_nav.dat). If all 3 exist → cust_def_data = 1, read_navdata2() (7187-7192).
  3. Else Resources/default data/earth_nav.dat, earth_fix.dat, earth_awy.dat; all 3 → cust_def_data = 2, read_navdata2() (7193-7226). Else nothing is loaded and cust_def_data stays 0.

read_navdata2 (7332), in this order:

  1. Custom Scenery/Global Airports/Earth nav data/earth_nav.dat (7403-7472; GA overrides of ILS-type rows). If it is missing, custom_ils_status = 0 (7474).
  2. Custom Data/user_nav.dat: written (open at 7478-7482) when custom_ils_status == 1, else read (7483-7536).
  3. Custom Data/airports.dat (7538-7603): comma-separated icao,lat,lon,ta,tl,magvar, parsed into local tables navdata_icao* (7383-7395) that are thrown away when the function returns. Dead code: nothing outside reads them. (Custom Data/runways.dat is assigned to a variable at 7382 but never opened.)
  4. <path>earth_nav.dat (file_name_path_nav, opened at 7353).
  5. <path>earth_fix.dat (7971-7972).
  6. <path>user_fix_georef.dat (8031, uses file_name_path, optional).
  7. <path>earth_hold.dat (8093, optional).

detect_apt_dat (8453): Log.txt (line 1 → log_line), Custom Scenery/scenery_packs.ini (builds custom_scenery[] = each pack's Earth nav data/apt.dat that exists, plus a byte-sum stamp; XP12 *GLOBAL_AIRPORTS* → Global Scenery/Global Airports/Earth nav data/apt.dat), then the four caches file_path2 .. "B738X_apt.dat", "B738X_rnw.dat", "B738X_gate.dat", "B738X_jetway.dat". Each cache must start with 4 header lines: log_line, log_line2 (scenery count), log_line3 (stamp), v1.1. Any mismatch → create_apt_rnw_dat2() + update_apt_dat = 1 (8622-8625).

create_apt_rnw_dat2 (8630): reads, in order, every custom_scenery[] apt.dat except Global Airports (database 0, 8792), then the Global Airports apt.dat (Custom Scenery/... if simDR_int_ver < 120000, else Global Scenery/...), then on XP11 only Resources/default scenery/default apt dat/Earth nav data/apt.dat (8792-8838). The first occurrence of an ICAO wins (later duplicates only fill in TA/TL that were 0; 8991-8994, 9324-9329). It writes the 4 caches (header + rows).

read_rnw_dat (9491) / read_apt_dat (9394): read the caches back (the first line is always skipped; header lines don't match the word counts).

read_awy_dat2 (9572): file_name_path_awy .. "earth_awy.dat".

On demand (see §4): Custom Data/CIFP/<ICAO>.dat or Resources/default data/CIFP/<ICAO>.dat.

Other readers outside a_fms: plugins/xlua/scripts/HUD_FIX/HGS_Airport_Data.lua (88, 300-327) parses apt.dat itself. plugins/zibomod/*/zibomod.xpl (binary) contains the strings B738X_gate/jetway/rnw/apt; per docs/M5C_FMS_SURVEY.md it also reads earth_fix/earth_nav for the ND map. Neither is covered here.

1.4 Caches Zibo writes and reads back#

File (in file_path2) Written by Row format (space-separated) Read back by
B738X_apt.dat create_apt_rnw_dat2 9378-9386 (all airports at the end) ICAO lat lon ta tl longest_rwy_m (6 words) read_apt_dat (rows with exactly 6 words)
B738X_rnw.dat 8932-8972, 9264-9304 (per airport, sorted) ICAO rwy lat_start lon_start lat_end lon_end length_m true_crs (8 words) read_rnw_dat (exactly 8 words)
B738X_gate.dat 8974-8980, 9306-9313 ICAO lat lon hdg name... not read by Lua (zibomod binary references the name)
B738X_jetway.dat 8982-8988, 9315-9321 ICAO lat lon not read by Lua
Custom Data/user_nav.dat read_navdata2 7607-7629, 7666-7756, 7964-7968 when custom_ils_status == 1 earth_nav.dat lines (header, then merged GA ILS rows, then 99) read_navdata2 (overrides) and check_navdata (first 2 lines)

All four B738X caches start with the 4-line header described above.


2. Global tables filled from navdata#

All rows are Lua arrays created as {true, true, ...} and then filled. Numbers come from tonumber (double). Strings stay as text. "Order" = order of insertion; nothing is sorted except runways inside an airport. Every *_num counter equals the row count.

2.1 earth_nav[1..earth_nav_num]: navaids and fixes (9 fields)#

idx meaning type source
1 kind: 1 VOR, 2 VOR-DME/VORTAC, 3 NDB, 4 FIX, 5 DME-only number row code (below)
2 latitude, deg (+N) number nav word 2 / fix word 1
3 longitude, deg (+E) number nav word 3 / fix word 2
4 ident (e.g. "SFO", "BOGEY") string nav word 8 / fix word 3
5 terminal area: airport ICAO or "ENRT" string nav word 9 / fix word 4
6 frequency as text: VOR/DME in 10 kHz ("11580" = 115.80), NDB in kHz ("362"); fixes "" string nav word 5
7 name, first word only (word 11) ; fixes "" string nav word 11
8 ICAO region code (e.g. "K2") string nav word 10 / fix word 5
9 VOR slaved variation, deg as in the file (NDB/DME/FIX = 0) number nav word 7 (VOR only)

Order: all navaids from earth_nav.dat (file order, codes 2/3/13 only), then all earth_fix.dat fixes (file order), then user_fix_georef.dat fixes.

Type 2 is set only when a code-12 row comes right after a code-3 row with the same ident (word 8) and frequency (word 5) (7888-7891). Whether that happens depends on the file's row order. The X-Plane files are not in this repo, so whether real data produces type 2 is unknown. Consumers treat 1 and 2 the same at 21871.

Consumers (only these functions read earth_nav): find_navaid 23030, find_navaid2 23569, wpt_exist2 49687 (kind == 4), navaid_exist2 49713 (kind ~= 4), dump_leg2 (debug). navaid_list[k][1..8] copies fields 1-8; navaid_list[k][25] = earth_nav[..][9] (23136, 23650). vor_declination = -navaid_list[1][25] is used for procedure radial legs (20433, 20537).

2.2 ils_nav[1..ils_nav_num]: localizers (10 fields)#

idx meaning type source (earth_nav.dat code 4 or 5)
1 ident (e.g. "ISFO") string word 8
2, 3 LOC antenna lat, lon number words 2, 3
4 runway ("28R"); must match the rnw_data designator format ("09", "09L") string word 11
5 true course, deg (see note) number word 7
6 frequency, 10 kHz (11030) number word 5
7 airport ICAO string word 9
8 region code string word 10
9 0 ILS/IGS (code 4 ILS*/IGS*), 1 LOC-only (code 5), 2 code 4 with LOC*, 3 LDA (code 4 LDA*) number code + word 12 prefix
10 magnetic course, deg, or -1 number word 7

Course decoding (7848-7854, 7873-7879): if word7 > 360 then [10] = floor(w7/360) and [5] = w7 - [10]. This is not w7 - [10]*360, so [5] is wrong in that case. It is harmless: consumers use [10] when [10] >= 0 and [5] + mag_variation_deg(lat,lon) only when [10] == -1 (find_id_app_data 13752-13756). Code 4 rows need word 12 starting with ILS, LOC, LDA or IGS (7839-7840). Code 5 accepts any word 12.

Consumers: find_navaid/find_navaid2 (linear over all ils_nav, 23329, 23861, only when no other match or mode 4), find_id_app_data 13672 (via idx_ils_icao; matches ICAO + runway des_runway_old_name, i.e. rnw_data[10]), is_ils 144800 (via idx_ils; ident + [6] compared with simDR_nav1_freq), B738_vor_sel.

2.3 gls_nav[1..gls_nav_num]: GLS / LPV (11 fields), earth_nav.dat codes 15 and 16 (7812-7837)#

idx meaning source
1 ident: code 15 word 8; code 16 word 12 (approach id, e.g. "W26A") string
2, 3 lat, lon words 2, 3
4 runway word 11
5 true course tonumber(sub(w7,4))
6 channel word 5 (number)
7 airport ICAO word 9
8 region word 10
9 glide path deg tonumber(sub(w7,1,3))/100
10 height word 4 (number)
11 0 GLS (code 15), 1 LPV (code 16)

Consumers: B738_gls_perf/B738_gls (per frame, §6), find_id_app_data (approach J = GLS, R = LPV; course + mag_variation_deg), find_navaid*, is_gls.

2.4 hold_nav[1..hold_nav_num]: published holds, from earth_hold.dat (9 fields; 8133-8147)#

[1] fix ident (w1), [2] region (w2), [3] inbound course, magnetic (w5), [4] leg time, minutes (w6), [5] leg distance, NM (w7), [6] turn 0 = L, 1 = otherwise (w8), [7] min alt ft (w9), [8] max alt ft (w10), [9] speed kt (w11). Rows need more than 10 words. Words 3 and 4 are not used. Consumers: find_hold_data 13844 (via idx_hold, ident + region), rte_add_hold 28933, rte_add_hold_wpt (26175-26208), sel_add_hold_wpt.

2.5 apt_data[1..apt_data_num]: airports (6 fields; read_apt_dat 9451-9479)#

idx meaning type
1 ICAO (4 chars, by construction of the cache) string
2, 3 lat, lon (datum; see §3.4) number
4 transition altitude, ft (0 if unknown) number
5 transition level, ft. Cache value < 1000 → ×100; "FLxxx" → xxx×100; else as is; 0 if unknown number
6 longest runway, m (integer) number

Order: order of first appearance in the apt.dat files. Consumers: apt_exist 49623 (sets icao_latitude, icao_longitude, icao_tns_alt, and icao_tns_lvl = [5], or [4]+1000 if [5]==0 and [4]~=0), apt_exist2 49661, airport_pos(2) 132055/132114, find_navaid* (airports as kind 9).

2.6 rnw_data[1..rnw_data_num]: runway ends (10 fields; read_rnw_dat 9545-9558)#

idx meaning type
1 airport ICAO string
2 designator "09", "09L", "27C" (2-digit, zero-padded). Overwritten by read_ref_data/read_des_data to the CIFP spelling when they differ (39225, 39289, 40487, 40548) string
3, 4 this end's lat, lon (apt.dat runway end point) number
5, 6 opposite end lat, lon number
7 length, m (great-circle-ish distance between the two ends, rounded) number
8 true course, deg (float; end 2 = end 1 + 180, not recomputed) number
9 scratch flag (0 at load; set to 1 by the CIFP matching code) number
10 original designator (as loaded; used to match ils_nav[4]) string

Order: airports in apt.dat order. Within an airport, both ends of each row 100, then sorted by key number + 0.3 (L) / + 0.7 (R) (C and none: +0), with an unstable exchange sort (8933-8963, 9265-9295). This order is the runway list order on the DEP/ARR pages (create_rnw_list 110280 does not sort).

Consumers: about 25 functions, all through idx_rnw[byte(icao)]: find_ref_rnw_data 13314, find_des_rnw_data 13418, ref/des_rnw_length_crs(2) 13872-14240, find_ref/des_runway 14267/14312, rte_add_sid2 30238 (RWxx leg: course [8] true, length [7], position = [3],[4] + 0.7×length along [8]), rte_add_app2 32635, create_rnw_list/update_rnw_list, nav_data_find 52270, rnw_exist 49739, find_gls_runway(2) 145049/145094, read_ref_data/read_des_data, 1L/3R CDU handlers. read_navdata2 also reads it while merging GA ILS rows.

2.7 awy_data[1..awy_data_num]: airway segments (6 fields; read_awy_dat2 9636-9668)#

[1] airway name (string; one row per name when word 11 is "A1-B2-C3"), [2] from fix ident (w1), [3] to fix ident (w4), [4] from region (w2), [5] to region (w5), [6] direction "N" (both), "F" (from→to only), "B" (to→from only) (w7). Words 3 and 6 (fix types) and 8-10 (high/low, base, top) are ignored, so the segment's fix type is lost. Lookups are by ident + region only. Rows need exactly 11 words (a CR on word 11 is stripped). The first line of the file is skipped.

Consumers: find_awy 9682 (collects every segment of the airway from the idx_awy bucket, removes duplicates, checks that the entry fix is on it), find_awy_path 9825 (greedy walk over those segments, O(k²), result depends on segment order), find_via_via 10088, dump_log.

2.8 Index tables (create_idx_table 8177)#

idx_earth_nav, idx_apt, idx_ils, idx_ils_icao, idx_gls, idx_gls_icao, idx_rnw, idx_awy and idx_hold all have the same shape:

idx_X[b]          for b = 48..57 ('0'-'9') and 65..90 ('A'-'Z')   (36 buckets)
idx_X[b][99999] = count k
idx_X[b][1..k]  = row index into the data table, in ascending row order (insertion order)

The key is string.byte of the first character of: earth_nav[4] (ident), apt_data[1], ils_nav[1] (ident) and ils_nav[7] (ICAO) for idx_ils_icao, gls_nav[1] / gls_nav[7], rnw_data[1] (ICAO), awy_data[1] (name), hold_nav[1] (ident). Rows whose key starts with any other character (lowercase, punctuation, empty) are not indexed and can never be found. A bucket stops at 99998 entries; further rows are silently dropped (8241 and the others).

Search pattern (everywhere): compute b = byte(first char of key), then a linear scan of idx_X[b][1..count], comparing strings (==). There is no binary search and no hash lookup. Examples:

  • find_navaid(nav_id, nav_icao, mode, reg_code, in_rte) 23030: scans idx_earth_nav[b] and matches ident; also ICAO ([5]) if given and region ([8]) if given. Then idx_apt[b] (airports), wptx, custom wpts, RW legs. Then sorts all matches by distance from a reference point (aircraft/origin/dest/find_lat). ils_nav/gls_nav are scanned linearly over the whole table only if nothing matched (or mode 4).
  • find_navaid2(..., nav_ndb, ...) 23569: the same, plus an NDB filter (nav_ndb == 2 → only kind 3; 1 → exclude kind 3). It is used for procedure fixes in dec_lat_lon2 (20215+) with ident = CIFP fix (field 5) and region = CIFP field 6, no ICAO.
  • apt_exist(icao) 49623: idx_apt[b], exact ICAO, first match.
  • find_ref_rnw_data 13314 / rnw_exist: idx_rnw[b], ICAO + designator.
  • find_awy 9682: idx_awy[b], airway name, all matches.
  • find_hold_data(id, rc) 13844: idx_hold[b], ident + region, first match.
  • find_id_app_data 13672: idx_ils_icao[b(ICAO)], then ICAO + runway.

So the provider must preserve: (a) string formats exactly (ident case, 2-digit runway designators, region codes), (b) the bucket structure and [99999] count, (c) for runways, the per-airport order.

2.9 Per-airport procedure tables: see §4.#


3. X-Plane file fields consumed, and their destinations#

3.1 earth_nav.dat (X-Plane 1150 format; read_navdata2 7586-7960)#

Header: lines are read until one contains "cycle " (7605-7629). airac_cycle = the 4 chars after it, and build = the 8 chars after "build ". If no line contains "cycle ", the whole file is consumed and no navaids load, airac_navdata stays "", and airways are skipped too.

A data row is only processed when it has 12 or more words (j > 11, 7810).

Code Stored? Destination
2 NDB yes earth_nav kind 3: w2 lat, w3 lon, w5 freq (kHz, string), w8 id, w9 ENRT/ICAO, w10 region, w11 name word 1; magvar 0
3 VOR yes earth_nav kind 1: same fields + w7 (slaved variation) → [9]
4 LOC (of ILS) yes, if w12 starts with ILS/LOC/LDA/IGS ils_nav (§2.2)
5 LOC-only yes ils_nav type 1
6 GS, 7/8/9 markers no (6 only feeds dead "skip" logic)
12 DME (paired) no row turns the VOR just before it into kind 2 (same id + freq)
13 DME standalone yes earth_nav kind 5
14 no
15 yes gls_nav type 0 (GLS)
16 yes gls_nav type 1 (LPV), id = w12
99 / short rows ignored

Words 4 (elevation) and 6 (range) are never stored. The "skip" path at 7893-7904 never fires, because old_icao is always set to "" for kinds 1/3/5.

GA / user_nav overrides (7666-7807): for codes 4-9 and 12 with more than 11 words, a row whose (w1, w8, w9, w10) matches a row in Custom Scenery/Global Airports/Earth nav data/earth_nav.dat (then Custom Data/user_nav.dat) is replaced by that row. When regenerating user_nav.dat, the GA runway (w11) is renamed to match rnw_data (±1 runway number, same suffix). For an MSFS provider this whole path is moot (no such files). Recommendation: B738DR_cust_nav_data = 0 and no GA file.

3.2 earth_fix.dat (X-Plane 1101) and user_fix_georef.dat (7970-8088)#

The first 3 lines are skipped. Rows need more than 4 words: w1 lat, w2 lon, w3 ident, w4 terminal area (ENRT or airport ICAO), w5 region → earth_nav kind 4 ([6]="", [7]="", [9]=0). w6 (waypoint type) is ignored.

3.3 earth_awy.dat (X-Plane 1100) (9572)#

The first line is skipped. Rows with exactly 11 words: w1 from id, w2 from region, [w3 from type], w4 to id, w5 to region, [w6 to type], w7 direction N/F/B, [w8 1 = low / 2 = high, w9 base, w10 top], w11 name(s) joined with -.

3.4 apt.dat (create_apt_rnw_dat2; only via the B738X caches)#

Row Use
1 (with more than 5 words) Starts a new airport. ICAO = w5 if its length is 4; lat/lon reset to "0". Rows 16 (seaplane base) and 17 (heliport) only close the previous airport (8907).
1302 icao_code X ICAO = X if its length is 4 (overrides w5)
1302 datum_lat/datum_lon Airport lat/lon, if not yet set
1302 transition_alt N TA (ft)
1302 transition_level FLnnn → nnn·100; number < 1000 → ·100; else as is
100 (more than 19 words) w9 / w18 end designators; w10-11 / w19-20 end lat/lon. Length = nd_calc_dist × 1852 (rounded); true course = nd_calc_brg(end1 → end2); end 2 course = +180. Displaced threshold and blastpad (w12-13, w21-22), width, surface: ignored. If the airport still has no lat/lon, end 2 of the first runway is used.
15 / 1300 gate/start positions (B738X_gate.dat only)
1500 jetways (B738X_jetway.dat only)
99 end of file: closes the last airport

Rows 101 (water) and 102 (helipad) are ignored. An airport without a 4-char ICAO is dropped. The B738X_rnw.dat block for an airport is written only if it has at least 2 runway ends (num_rnw > 1).

3.5 earth_hold.dat (8090-8150)#

The first 3 lines are skipped. Rows with more than 10 words: w1 id, w2 region, w5 inbound magnetic course, w6 time (min), w7 distance (NM), w8 L/R, w9 min alt, w10 max alt, w11 speed.

3.6 CIFP <ICAO>.dat: see §4.3.#


4. Procedures (SID/STAR/approach), from CIFP#

4.1 When they load: on demand, synchronously, one airport per call#

Each opener does file_navdata = io.open("Custom Data/CIFP/"..ICAO..".dat") (or "Resources/default data/CIFP/", by cust_def_data). If that fails and the airport is in apt_data, the procedure counts are set to 0 (the airport is still accepted). Otherwise it calls read_ref_data(ICAO, in_rte) (origin) or read_des_data(ICAO, in_rte) (destination), which read the global handle file_navdata line by line. The caller then closes it. in_rte: 0 = RTE 1, 1 = RTE 2.

Call sites (54 opens: 27 Custom Data + 27 default data):

  • ref_apt_exist 52831 / des_apt_exist 52882, called from CDU 1L/1R handlers (B738_fmc1/2_1L/1R_CMDhandler): when an ORIGIN/DEST is typed on RTE page 1.
  • read_ref_list_icao 52599 / read_des_list_icao 52723: DEP/ARR page selection (CDU 1L/1R/3L/3R/2R/4R handlers) and B738_flight_phase3 (10 sites, e.g. 133260: after landing the destination becomes the origin).
  • import_fms_v11 15397, import_fmx 16801, use_import_data, Datalink_rte: flight plan load / uplink (origin + destination).
  • load_fmc (saved FMC state, 34897).
  • B738_fmc1/2_2L_CMDhandler (62775): POS INIT ref airport (open test only).
  • read_gls_runway_height 144919: rescans the file for one RWY: record (threshold elevation and lat/lon).

read_ref_data and read_des_data both start by clearing their tables, so the last call wins. The _1/_2 copies hold the per-route state.

4.2 Data structures#

Origin (read_ref_data 38756), working set, then copied by copy_from_ref_data(in_rte) 39327 into *_1 (in_rte 0) or *_2 (in_rte 1), and by copy_from_rte_sid 39673 into rte_sid_1/_2. copy_to_ref_data 39500 / copy_to_rte_sid 39703 copy them back.

Table Row Filled from
ref_data_icao string the ICAO
ref_data_rwy[1..ref_rwy_num] {rwy "08L"/"08 " (chars 7-9 of the RWY line), threshold elev string (field 4), used 0/1, lat, lon, true crs} (fields 4-6 copied from the matched rnw_data) RWY: lines
ref_data_sid[1..sid_num] {sid_id, "RWxxx"/"ALL"}: one row per distinct (id, runway) in file order SID: lines with field 4 RW* or ALL
ref_data_sid_tns[1..sid_tns_num] {sid_id, trans_id} (field 4 with whitespace removed, non-empty) other SID: lines
ref_data_star / ref_data_star_tns (star_num, ref_data_star_tns_n) as SID STAR:
ref_data_app[1..ref_data_app_n] approach id string, sorted with rnw_number_weight 40585 (runway ×10, +3 L, +6 R, + type 0 I, .1 J, .2 L, .3 R, .4 N/Q, .5 D/S, .6 V, .7 other) APPCH: with field 2 ≠ "A"
ref_data_app_tns[1..ref_data_app_tns_n] {app_id, trans_id, first fix} APPCH: with field 2 == "A"
ref_ed_app[1..ref_ed_app_n] {app_id, fix} where field 9 == "E F" exactly
rte_sid[1..rte_sid_num] leg rows (25 fields, §4.4), all SID lines (runway parts + transitions), file order SID:

Destination (read_des_data 39804): des_data_icao; des_data_rwy[1..des_rwy_num] = {"RW08L"/"RW08 " (chars 5-9), "", "", ILS ident (field 6), elev (field 4), matched 0/1}; data_des_star(_n), data_des_star_tns(_n), data_des_app(_n) (sorted as above), data_des_app_tns(_n) (3 fields), ed_app / ed_app_num (field 9 longer than 3 chars and ending in F; not copied); rte_star[1..rte_star_num] and rte_app[1..rte_app_num] (leg rows). Copies: copy_from_des_data → des_data_rwy_1/2, data_des_star_1/2...; copy_from_rte_star 40877 → rte_star_1/2; copy_from_rte_app 40937 → rte_app_1/2. Counters: *_num_1/_2, *_n_1/_2.

The origin's approaches and STARs are listed (ARR page for the origin), but no rte_app/rte_star legs are built from the origin file.

Both readers also rename rnw_data[..][2] for the airport to the CIFP spelling (exact match, else same suffix and within 30°, else the unique same number) and fill ref_data_rwy[4..6].

Consumers: create_sid_list 110538, create_star_list 111084, create_app_list 111462 (DEP/ARR lists, in the order above: no extra sort), rte_add_sid2 30115, rte_add_star2 31096, rte_add_app2 31831 (legs → legs_data2b), dec_lat_lon2 20215 (geometry: resolves fixes with find_navaid2(id, "", mode, region, ndb_flag)), find_sid_app_idx, find_id_app_data.

4.3 CIFP line format consumed#

Each line is split with string.gmatch(line, "[^,]+") (38810, 39856). Empty fields are dropped, so X-Plane CIFP blank fields must be padded with spaces (as X-Plane's files are) or every later index shifts. read_gls_runway_height splits on , and ;.

RWY: line, e.g. RWY:RW08L, , ,00011, ,IOSL,1, ;N38415050,W121353024,0000;: f1 RWY:RW08L (designator = chars 5-9, padded to 5), f4 threshold elevation (ft), f6 localizer ident. read_gls_runway_height also uses f9 N38415050 (N/S, DD, MM, SS, hundredths) and f10 W121353024.

SID:/STAR:/APPCH: lines. The record number is the number after the colon (SID:010 → 10, stored in [25]):

field Zibo's use
1 SID:nnn / STAR:nnn / APPCH:nnn
2 route type. APPCH "A" = approach transition
3 procedure id
4 runway (RW08L, RW08B = all parallels, ALL) or transition id
5 fix ident (blank allowed)
6 fix region code
7, 8 fix section/subsection. (P,N) or (D,B) → NDB fix
9 waypoint description, 4 chars: char 2 Y = fly-over; APPCH char 4 F = FAF, char 3 M = missed approach point, char 3 A/B/C/D = IAF/IF, char 4 I
10 turn direction L / R / E
11 RNP
12 path terminator (IF, TF, CF, DF, FA, FC, FD, FM, CA, CD, CI, CR, AF, RF, VA, VD, VI, VM, VR, PI, HA, HF, HM)
14, 15 recommended navaid ident / region
18 arc radius
19 theta (bearing from rec navaid, tenths of a degree)
20 rho (distance from rec navaid, tenths of NM)
21 magnetic course, tenths of a degree
22 route / hold distance (tenths) or time
23 altitude descriptor (+, -, B, blank, G, H, I, J, V, X, Y)
24, 25 altitude 1, 2 ("05000" or "FL180")
27 speed descriptor (+, -, blank)
28 speed limit, kt
29 vertical path angle

Fields 13, 16, 17 and 26 are not used.

4.4 Leg row (rte_sid, rte_star, rte_app: 25 fields, values kept as the raw CIFP strings#

unless noted; SID 38966-39050, STAR 40006-40090, APP 40255-40362; the FD-inserted rows at 38874, 39914, 40142)

idx meaning
1 procedure id (f3)
2 runway / transition (f4; for SID transitions, with whitespace removed)
3 display name, dec_name_cifs(f5, f12, f14, f19, f22, f24) 38645: (VECTOR) for VM/FM, (alt) for xA, (INTC) for xI and blank CF, NAV123 for xR, NAV-12 for xD/FC, otherwise the fix id
4 speed (f28)
5 altitude 1. For B: f25; for G/H/X/Y/V: f25; otherwise f24
6 altitude type: B→"B", G/H/X/Y/V/I→" ", J→"+", otherwise f23
7 f19 (theta)
8 f29 (VPA)
9 turn: -1 none, 2 L, 3 R/E; hold legs (f12 starts with H): 0 L, 1 R
10 f22 (distance/time)
11 f21 (magnetic course)
12 f6 (fix region)
13 f14 (recommended navaid)
14 f15 (its region)
15 path terminator (f12), or "IF" for an inserted leg
16 fly-over 0/1 (1 for CA/VA/FA/FD/VM, or f9 char 2 == Y)
17 APPCH only: 0, 1 FAF, 2 MAP, 3 IAF/IF, 5 I
18 f20 (rho)
19 f18 (arc radius)
20 raw fix id (f5)
21 f11 (RNP)
22 1 if NDB fix (f7/f8 = P/N or D/B)
23 altitude 2 (f24 for B; 0 for G/H/X/Y/V/I/J; otherwise f25)
24 speed descriptor (f27)
25 record number (+1 per inserted leg in STAR/APP)

STAR/APP only (39899-39905, 40127-40133): before an FD leg whose fix differs from the previous row's name (or that is first), an extra IF leg at that fix is inserted. This is disabled for SIDs (fd_compare stays false; the code is commented out at 38859-38865).

Downstream, rte_add_*2 parse [4], [5], [23] with tonumber ("FLnnn" → nnn·100), [10]/[11] as tenths (/10), and map [6] to +=43, -=45, B=41, otherwise 0 (30490-30530, 30755-30810).


5. AIRAC, magnetic variation, Custom Data vs default data#

  • AIRAC: only from the earth_nav.dat header (cycle NNNN, build YYYYMMDD). find_airac_cycle2 8405 computes the cycle's dates from 5 Jan 2017 in 28-day steps (cycles before 17xx → ""). Effects: IDENT page text; NAV DATA OUT OF DATE (UTC today after the cycle's end date) / CORRUPTED NAVDATA (empty or short string) in B738_fmc_msg 134926; airac_navdata ~= "" gates airway loading (170642). The provider must set airac_cycle, airac_navdata and airac_navdata_active (or call find_airac_cycle2(cycle) itself).
  • Magnetic variation sources:
    • mag_variation_deg(lat, lon) / _rad 18161-18177: writes zibomod/lat_mag_var, lon_mag_var, calc_mag_var = 1, and reads zibomod/mag_var back in the same call (zibomod computes it synchronously in X-Plane). 137 call sites (runway courses for RWxx legs 30294, ref/des_rnw_length_crs 13915, ILS course when [10] == -1, GLS course, leg courses in dec_lat_lon2...). In the port, zibomod/mag_var is the aircraft-position variation (fms.cpp:353-356, true − magnetic heading), not the requested point's. A position-specific mag var service is a separate gap.
    • simDR_mag_variation (sim/flightmodel/position/magnetic_variation, 3473): 176 uses (aircraft position).
    • Navdata-embedded: VOR slaved variation (earth_nav[9], for radial legs), ILS magnetic course (ils_nav[10]), hold inbound course (magnetic), CIFP course fields (magnetic, tenths). rnw_data[8] is true (computed from coordinates). Custom Data/airports.dat magvar is read and discarded.
  • Airport elevation: not from navdata. Lua writes laminar/B738/find_icao + zibomod/find_icao_cmd = 1 and expects zibomod to set find_icao_cmd = 2 and laminar/B738/find_icao_height (m) synchronously (16085-16090, 16459-16470). Nothing in wasm/src implements this (grep), so the origin/destination elevation is 0 or 2500 in the port today. The CIFP RWY threshold elevations (ref_data_rwy[2], des_data_rwy[5]) are a separate path.
  • Custom Data vs default: cust_def_data chooses the directory for earth_* files, user_fix_georef, earth_hold and CIFP. user_nav.dat and airports.dat are always read from Custom Data/. If neither set is complete (3 files), no navaids/fixes/airways/holds load and CIFP is never opened. Airports and runways still load (from the apt.dat caches).

6. Data volumes and performance#

From the code: rows are 6-10 field Lua tables. The index adds one integer entry per row per index (ILS and GLS have 2 indexes).

Measured in this repo (docs/M5C_FMS_SURVEY.md §3.3, synthetic set from tools/fms_spike/gen_nav.py: 250k fixes, 25k navaids, 120k airway segments): about 275k earth_nav rows; +105 MB Lua heap on PUC 5.1 (64-bit); parse 15.5-16.2 s on PUC 5.1 (character-by-character tokenizing). Real X-Plane/Navigraph counts are not in the repo. Order of magnitude to expect (not derived from the code; verify): fixes ~250-300k, navaids (2/3/13) ~20-25k, localizers ~5k, airway lines ~100-150k (more awy_data rows from hyphenated names), airports with a 4-char ICAO ~30-40k, runway ends ~2 × runways. An identifier's first letter is not uniform, so some buckets (for example I for idx_ils, and common fix initials) are many times larger than the average.

Per-frame scans (from after_physics 171531):

  • B738_gls_perf 145170, every frame: scans gls_nav 15 rows per frame (1000 per frame for the first two passes, first_time_gls < 2). Distance test (40 NM, 8° box). Builds gls_page1/2.
  • B738_gls 145267 (if first_time_gls > 0 and MMR on): iterates only the nearby page list.
  • B738_vor_sel 146375 → is_ils 144800, every frame while a NAV receiver is tuned to a localizer frequency (up to 4 call sites, 146469-146679): a linear scan of idx_ils[byte(ident)]. Most localizer idents start with I, so this bucket is a large share of all localizers. This is the main O(n)-per-frame navdata cost.
  • Event-driven, but large: find_navaid/find_navaid2 (bucket scan, plus a linear scan of all ils_nav and gls_nav when nothing matched), called once per procedure leg during dec_lat_lon2 and route (re)calculation. find_awy + find_awy_path per airway entry (O(k²) in the airway's segments). find_id_app_data on flight phase events (134080, 134401) and in B738_calc_rte. Each access to a global table name costs one namespace_read metamethod call (the survey profiled 62 % of FMS time in namespace_read).

7. What an MSFS provider must supply (summary)#

Lua target Needed per row
earth_nav + earth_nav_num VOR (kind 1/2), NDB (3), standalone DME (5), waypoints (4): lat, lon, ident, terminal area (ENRT or airport ICAO), freq text (VOR/DME 10 kHz, NDB kHz), first word of the name, region, VOR slaved variation
ils_nav + count LOC/ILS/LDA/IGS: ident, lat, lon, runway (rnw_data format), true course, magnetic course (or -1), freq (number, 10 kHz), ICAO, region, type 0-3
gls_nav + count GLS / SBAS LPV paths (MSFS equivalent: unknown; 0 rows disables the feature cleanly)
hold_nav + count published holds: fix id, region, inbound magnetic course, time, distance, turn, min/max alt, speed
apt_data + count ICAO, lat, lon, TA ft, TL ft, longest runway m
rnw_data + count ICAO, designator "09"/"09L", both end lat/lon, length m, true course, 0, designator again; per-airport sort order
awy_data + count airway name, from/to ident, from/to region, direction N/F/B
idx_* rebuild exactly as create_idx_table does (or just call it)
airac_cycle, airac_navdata, airac_navdata_active, cust_def_data (1 or 2, so CIFP opens are attempted)
CIFP per airport either <ICAO>.dat text (via a real or virtual file) in the exact comma layout of §4.3, or direct fills of the §4.2 tables (including the _1/_2 copies and the rnw_data[2] renaming)

Unknowns (not resolvable from this code alone): the row order of the real earth_nav.dat (affects VOR kind 2); the sign convention of X-Plane's VOR slaved variation versus MSFS's VOR magvar; how MSFS represents GLS/LPV paths; real-world row counts.

VIA entries wait for the airway (build 59)#

An airway Zibo's find_awy does not have, entered as a VIA (via_add / via_chg), is held: no INVALID ENTRY, the scratchpad keeps it, the provider follows the airway from the fix. Once it is done (lu737_awy_done, found or not) or after 8 s, the entry runs again if the scratchpad still holds it: taken, or Zibo's INVALID ENTRY.