Difference between revisions of "CSV data format"

From bcmeter.org
Jump to navigation Jump to search
(Fix ATN formula: ATN = -100*ln(Sen/Ref), dimensionless; sigma enters BC not ATN (verified vs firmware measure.cpp))
(Consistency pass 2: airflow range via Configuration, multi-wavelength as in-development, ESP32/Pi CSV filename split, legacy sensor wording)
 
Line 21: Line 21:
=== Filename Convention ===
=== Filename Convention ===


* Active session: <code>YY-MM-DD_HHMMSS.csv</code> (based on session start time)
* Active session: <code>YY-MM-DD_HHMMSS.csv</code> on ESP32 builds, <code>DD-MM-YY_HHMMSS.csv</code> on Raspberry Pi builds (based on session start time)
* Fallback (no RTC sync): <code>nosync_HHMMSS.csv</code> (based on uptime)
* Fallback (no RTC sync): <code>nosync_HHMMSS.csv</code> (based on uptime)


Line 54: Line 54:
|}
|}


==== Multi-Wavelength Columns (optional, if hardware installed) ====
==== Multi-Wavelength Columns (in development, not in shipped hardware) ====


Same structure as 880nm, repeated for each wavelength:
Same structure as 880nm, repeated for each wavelength:

Latest revision as of 20:47, 8 June 2026

CSV Data Format

bcMeter stores measurement data as CSV files. This page documents the file format and column definitions.

File Format

Property Value
Delimiter Semicolon (;)
Encoding UTF-8
Decimal separator Dot (.)
Line ending LF (\n)
First row Column headers

Filename Convention

  • Active session: YY-MM-DD_HHMMSS.csv on ESP32 builds, DD-MM-YY_HHMMSS.csv on Raspberry Pi builds (based on session start time)
  • Fallback (no RTC sync): nosync_HHMMSS.csv (based on uptime)

Column Definitions

Core Columns (always present)

Column Type Unit Description
bcmDate string DD-MM-YY Measurement date
bcmTime string HH:MM:SS Measurement time
bcmRef_880nm float V Reference channel voltage (880nm LED through clean filter)
bcmSen_880nm float V Sensor channel voltage (880nm LED through sample spot)
bcmATN_880nm float Attenuation at 880nm = −100·ln(Sen/Ref), dimensionless. The absorption cross-section σ enters the BC concentration, not ATN.
BCngm3_unfiltered_880nm float ng/m³ Real-time BC concentration (unsmoothed)
BCngm3_880nm float ng/m³ BC concentration with rolling average applied
relativeLoad float % Filter loading (0 = clean, 100 = saturated)
AAE float Ångström Absorption Exponent (requires multi-wavelength)
Temperature float °C Internal device temperature
airflow float L/min Measured airflow rate

Multi-Wavelength Columns (in development, not in shipped hardware)

Same structure as 880nm, repeated for each wavelength:

  • bcmRef_520nm, bcmSen_520nm, bcmATN_520nm, BCngm3_unfiltered_520nm, BCngm3_520nm
  • bcmRef_370nm, bcmSen_370nm, bcmATN_370nm, BCngm3_unfiltered_370nm, BCngm3_370nm

Environmental Columns (optional, sensor-dependent)

Column Type Unit Description
humidity float % Relative humidity (BME280/SHT4x)
hPa float hPa Atmospheric pressure (BME280)
PM2.5 float µg/m³ Particulate matter <2.5µm (SPS30)
PM10 float µg/m³ Particulate matter <10µm (SPS30)

GPS Columns (optional, if GPS module connected and has fix)

Column Type Unit Description
lat float degrees Latitude (WGS84)
lon float degrees Longitude (WGS84)
altitude float m GPS altitude above sea level

Metadata Columns

Column Type Description
notice string System notices or warnings per data point
sampleDuration float Actual measurement duration for this sample (seconds)
notes string User-added annotations

Example Data

bcmDate;bcmTime;bcmRef_880nm;bcmSen_880nm;bcmATN_880nm;BCngm3_unfiltered_880nm;BCngm3_880nm;relativeLoad;AAE;Temperature;airflow
04-03-26;19:37:09;0.9679;0.9321;3.76;1686;1686;0;0;0;0.4659
04-03-26;19:42:09;0.9745;0.9335;4.3;4028.5;1686;0;0;0;0.3088
04-03-26;19:47:09;0.9752;0.9315;4.59;2371.4;1830.2;0;0;0;0.2832

Import Notes

  • Excel: Use "Text Import Wizard" and set delimiter to semicolon. The date format DD-MM-YY may need manual conversion.
  • Python/Pandas: pd.read_csv('file.csv', sep=';')
  • R: read.csv2('file.csv') (uses semicolon by default)