Difference between revisions of "CSV data format"
Jump to navigation
Jump to search
(Wiki update: sync with manual and dev docs (2026-03)) |
(Consistency pass 2: airflow range via Configuration, multi-wavelength as in-development, ESP32/Pi CSV filename split, legacy sensor wording) |
||
| (One intermediate revision by the same user not shown) | |||
| Line 21: | Line 21: | ||
=== Filename Convention === | === Filename Convention === | ||
* Active session: <code>YY-MM-DD_HHMMSS.csv</code> (based on session start time) | * Active session: <code>YY-MM-DD_HHMMSS.csv</code> on ESP32 builds, <code>DD-MM-YY_HHMMSS.csv</code> on Raspberry Pi builds (based on session start time) | ||
* Fallback (no RTC sync): <code>nosync_HHMMSS.csv</code> (based on uptime) | * Fallback (no RTC sync): <code>nosync_HHMMSS.csv</code> (based on uptime) | ||
| Line 39: | Line 39: | ||
| <code>bcmSen_880nm</code> || float || V || Sensor channel voltage (880nm LED through sample spot) | | <code>bcmSen_880nm</code> || float || V || Sensor channel voltage (880nm LED through sample spot) | ||
|- | |- | ||
| <code>bcmATN_880nm</code> || float || — || Attenuation at 880nm = | | <code>bcmATN_880nm</code> || float || — || Attenuation at 880nm = −100·ln(Sen/Ref), dimensionless. The absorption cross-section σ enters the BC concentration, not ATN. | ||
|- | |- | ||
| <code>BCngm3_unfiltered_880nm</code> || float || ng/m³ || Real-time BC concentration (unsmoothed) | | <code>BCngm3_unfiltered_880nm</code> || float || ng/m³ || Real-time BC concentration (unsmoothed) | ||
| Line 54: | Line 54: | ||
|} | |} | ||
==== Multi-Wavelength Columns ( | ==== Multi-Wavelength Columns (in development, not in shipped hardware) ==== | ||
Same structure as 880nm, repeated for each wavelength: | Same structure as 880nm, repeated for each wavelength: | ||
Latest revision as of 20:47, 8 June 2026
CSV Data Format
bcMeter stores measurement data as CSV files. This page documents the file format and column definitions.
File Format
| Property | Value |
|---|---|
| Delimiter | Semicolon (;)
|
| Encoding | UTF-8 |
| Decimal separator | Dot (.)
|
| Line ending | LF (\n)
|
| First row | Column headers |
Filename Convention
- Active session:
YY-MM-DD_HHMMSS.csvon ESP32 builds,DD-MM-YY_HHMMSS.csvon Raspberry Pi builds (based on session start time) - Fallback (no RTC sync):
nosync_HHMMSS.csv(based on uptime)
Column Definitions
Core Columns (always present)
| Column | Type | Unit | Description |
|---|---|---|---|
bcmDate |
string | DD-MM-YY | Measurement date |
bcmTime |
string | HH:MM:SS | Measurement time |
bcmRef_880nm |
float | V | Reference channel voltage (880nm LED through clean filter) |
bcmSen_880nm |
float | V | Sensor channel voltage (880nm LED through sample spot) |
bcmATN_880nm |
float | — | Attenuation at 880nm = −100·ln(Sen/Ref), dimensionless. The absorption cross-section σ enters the BC concentration, not ATN. |
BCngm3_unfiltered_880nm |
float | ng/m³ | Real-time BC concentration (unsmoothed) |
BCngm3_880nm |
float | ng/m³ | BC concentration with rolling average applied |
relativeLoad |
float | % | Filter loading (0 = clean, 100 = saturated) |
AAE |
float | — | Ångström Absorption Exponent (requires multi-wavelength) |
Temperature |
float | °C | Internal device temperature |
airflow |
float | L/min | Measured airflow rate |
Multi-Wavelength Columns (in development, not in shipped hardware)
Same structure as 880nm, repeated for each wavelength:
bcmRef_520nm,bcmSen_520nm,bcmATN_520nm,BCngm3_unfiltered_520nm,BCngm3_520nmbcmRef_370nm,bcmSen_370nm,bcmATN_370nm,BCngm3_unfiltered_370nm,BCngm3_370nm
Environmental Columns (optional, sensor-dependent)
| Column | Type | Unit | Description |
|---|---|---|---|
humidity |
float | % | Relative humidity (BME280/SHT4x) |
hPa |
float | hPa | Atmospheric pressure (BME280) |
PM2.5 |
float | µg/m³ | Particulate matter <2.5µm (SPS30) |
PM10 |
float | µg/m³ | Particulate matter <10µm (SPS30) |
GPS Columns (optional, if GPS module connected and has fix)
| Column | Type | Unit | Description |
|---|---|---|---|
lat |
float | degrees | Latitude (WGS84) |
lon |
float | degrees | Longitude (WGS84) |
altitude |
float | m | GPS altitude above sea level |
Metadata Columns
| Column | Type | Description |
|---|---|---|
notice |
string | System notices or warnings per data point |
sampleDuration |
float | Actual measurement duration for this sample (seconds) |
notes |
string | User-added annotations |
Example Data
bcmDate;bcmTime;bcmRef_880nm;bcmSen_880nm;bcmATN_880nm;BCngm3_unfiltered_880nm;BCngm3_880nm;relativeLoad;AAE;Temperature;airflow 04-03-26;19:37:09;0.9679;0.9321;3.76;1686;1686;0;0;0;0.4659 04-03-26;19:42:09;0.9745;0.9335;4.3;4028.5;1686;0;0;0;0.3088 04-03-26;19:47:09;0.9752;0.9315;4.59;2371.4;1830.2;0;0;0;0.2832
Import Notes
- Excel: Use "Text Import Wizard" and set delimiter to semicolon. The date format DD-MM-YY may need manual conversion.
- Python/Pandas:
pd.read_csv('file.csv', sep=';') - R:
read.csv2('file.csv')(uses semicolon by default)