===========================
Pymepix Postprocessing
===========================

Raw data acquired from the camera can be processed using the command line or triggered from the `PymepixViewer`.

To process from the command line:

.. code-block:: console

    pymepix post-process -f FILE -o OUTPUT_FILE [-t TIMEWALK_FILE] [-c CENT_TIMEWALK_FILE] [-n NUMBER_OF_PROCESSES]

The output is an HDF5 file containing the following possible datagroups:

- **centroided**: Contains centroided data with datasets: "trigger nr", "x", "y", "tof", "tot avg", "tot max", and "clustersize".  

  - **trigger nr**: Event number

  - **x/y**: Centroid coordinates

  - **tof**: Time-of-flight (time-of-arrival corrected for timewalk)

  - **tot avg**: Average TOT across cluster voxels

  - **tot max**: Maximum TOT

  - **clustersize**: Number of voxels in the cluster

- **raw**: Event voxel data synchronized to the first trigger with datasets: "trigger nr", "x", "y", "tof", and "tot".

- **timing/timepix**: Contains "trigger nr", "timestamp", and "event trigger" datasets:

  - **timestamp**: Nanosecond resolution timestamps from the camera timer

  - **event trigger**: Trigger time in seconds since acquisition start

  - **trigger nr**: True trigger number as an unsigned integer. If a trigger doesn't contain any events, the "trigger nr" is still increased by one.

- **triggers**: May contain "trigger1" and "trigger2" subgroups, each with a "time" dataset.
  The number should correspond to "timing/timepix/event trigger" and can be used to align the two datasets.
  Rising edges carry a positive sign whereas the falling edges are denoted with a negative sign:

  - **trigger1**: Camera times for rising and falling edges for TDC1 in the camera time.

  - **trigger2**: Camera times for rising and falling edges for TDC2 in the camera time.

HDF5 file structure example:

.. code-block:: text

    <HDF5 file "ion_2595_20240130-1443.hdf5" (mode r)>
    ├── centroided
    │   ├── clustersize
    │   ├── tof
    │   ├── tot avg
    │   ├── tot max
    │   ├── trigger nr
    │   ├── x
    │   └── y
    ├── raw
    │   ├── tof
    │   ├── tot
    │   ├── trigger nr
    │   ├── x
    │   └── y
    ├── timing
    │   └── timepix
    │       ├── event trigger
    │       ├── timestamp
    │       └── trigger nr
    └── triggers
        ├── trigger1
        │   └── time
        └── trigger2
            └── time

To retrieve data from the HDF5 file into a Pandas DataFrame:

.. code-block:: python
    :linenos:

    import pathlib
    import pandas as pd
    import numpy as np
    import h5py

    def get_tpx_data(fname: pathlib.PosixPath) -> pd.DataFrame:
        '''Retrieve Timepix data from HDF5 file'''
        try:
            with h5py.File(fname, "r") as f:
                raw_data = pd.DataFrame(
                    np.column_stack((
                        f["raw/trigger nr"][:],
                        f["raw/tof"][:] * 1e6,
                        f["raw/tot"][:],
                        f["raw/x"][:],
                        f["raw/y"][:],
                    )),
                    columns=("nr", "tof", "tot", "x", "y"),
                )
                
                cent_data = pd.DataFrame(
                    np.column_stack((
                        f["centroided/trigger nr"][:],
                        f["centroided/tof"][:] * 1e6,
                        f["centroided/tot max"][:],
                        f["centroided/x"][:],
                        f["centroided/y"][:],
                        f['centroided/clustersize'][:]
                    )),
                    columns=("nr", "tof", "tot", "x", "y", "size"),
                )
                
            return raw_data, cent_data
        except:
            print(f'Error: Could not read data from "{fname}"')