Skip to content

Changelog

All notable changes to this project will be documented in this file. For each version important additions, changes and removals are listed here.

The format is inspired from Keep a Changelog and the versioning aims to respect Semantic Versioning.

[v0.14.3] Fix Pypi Release - 2024-04-24

Added

  • Add new table changed_dso_assignment #510
  • Add deprecation warning for MaStRMirror and MaStRDownload #492

Changed

  • Change from setup.py to pyproject.toml based installation #513

Removed

  • Remove outdated and deprecated dependencies #506

[v0.14.2] Maintenance - 2024-04-10

Changed

  • Fix and add URLs of example projects in readme #481
  • No longer require web scraping for bulk download #488
  • Replace deprecated pandas map function #491
  • Fix the handling of corrupted xml syntax in the downloaded files #494
  • Implement relevant API WSDL Patchnotes V24.1.128 #499

Removed

  • Remove unused Docker File #501

[v0.14.1] Hotfix - 2024-01-17

Changed

  • Change data type of NetzbetreiberpruefungStatus to string #483

[v0.14.0] Shiny New Documentation Page - 2023-11-22

Added

  • User-defined output path for csv, xml, database #402
  • Add date=existing parameter to Mastr.download #452
  • Update documentation and release it on mkdocs #460
  • Replace values in ArtDerFlaecheIds with their entries from katalogwerte #464
  • Add a Mastr.translate method for english translation ##471

Changed

  • Using sphinx version <7 to build documentation #454
  • Get missing column names in xml download in a robust way #470
  • Adapt column names to the API web service update #472

Removed

  • Delete on push for github workflow #445

[v0.13.2] Hotfix - 2023-08-07

Changed

  • Changed the name of the bulk tables for technology=gsgk #456

[v0.13.1] Hotfix - 2023-04-11

Added

  • Add new table and new columns to the data model #440

[v0.13.0] Maintenance release - 2023-02-16

Added

  • Add a workflow_dispatch to run CI pipelines from a button click #389
  • Update version number with bump2version #407
  • Add usage examples to Readme #419
  • Check if the zipped file is corrupted #422
  • Add Issue and PR Templates #428

Changed

  • Repair the csv export #384
  • Refactor csv export for better maintainability #401
  • Replace numeric value in hydro_extended #392
  • Fix DB query errors when using SQLAlchemy v2.0 #405
  • New columns introduced #409
  • Replace numeric value in combustion #413
  • Filenames.yml is created everytime Mastr class is called #415
  • Omit connectionless DB query (SQLAlchemy v2.0 compatibility) #421

Removed

  • Option to filter & export by parameter StatistikFlag #401
  • Removed unused package pynodo from dependencies #424
  • Removed feature for dockered databases #426

[v0.12.1] Patch release - 2022-11-15

Added

  • Add workflows to release on PyPI and Test-PyPI #375

Changed

  • Change error handling for missing tables in postgres #370

[v0.12.0] Minor Release - 2022-11-07

Added

  • New table deleted_units #360
  • Constants.py for central hardcoded variables #345

Changed

  • Handle case for empty list #362
  • Fix bug occuring for permit and location download from the bulk download source #358
  • Update documentation for readthedocs #356
  • Update release procedure #355
  • Update README.md to README.rst #354
  • Technology parameter is renamed to data for better comprehension #337
  • Update ORM and documentation according to wsdl patch V1.2.87 #352
  • Date parameters are united into one parameter #353
  • Dataversion folder entails export date#347

Removed

  • Removed code artefacts #335

[v0.11.7] Patch - Hotfix - 2022-08-25

Changed

  • Fix dependencies in setup.py #341

[v0.11.6] Patch - Hotfix - 2022-08-17

Changed

  • Repair technology paramter in API download #327

[v0.11.5] Patch - Repair package - 2022-08-16

Added

  • Pre-commit hook for Black formatting #312

Changed

  • Use exclusively sqlalchemy for dialect-free implementation #289
  • API Download: Linked units in table permit are written in comma seperated string #302
  • API Download: Repaired Location download #303
  • CSV export: Tables with mastr_data about power units are exported in chunks #306
  • API Download: User credentials are now tested before the download starts #309

Removed

  • Delete the MastrDownloadFactory class #292

[v0.11.4] Patch - Hotfix - 2022-07-08

Added

Changed

  • Move function cleaned_data to data_io #284

[v0.11.3] Patch - A data release - 2022-07-07

Added

  • Add warning message if the bulk download speed falls below a certain limit #256
  • Add engine parameter for master class initialisation #270

Changed

  • Refactor code and restructure modules #273
  • Fix combustion mapping #253
  • Update bulk parsing order #257
  • Apply csv method to all tables #275

[v0.11.2] Patch - Patch the package - 2022-05-17

Changed

  • Update readme.md to improve PyPi release #249
  • Rename branches dev -> develop and master -> production

[v0.11.1] Patch - Make it a package - 2022-05-16

Added

  • Add files and metadata for PyPi release #237

[v0.11.0] Unreleased - Forces unite - 2022-05-16

The code becomes a python package and will be available on pypi.org Additionally, a new datasource was identified and can be used: the xml bulk download. The API was updated to the newest version and the data model was adapted.

Added

  • The class :class:open_mastr.mastr.Matr was introduced as the entrypoint for users, the API download was included in this entrypoint #203
  • A method for downloading and parsing the xml dump from the MaStR website was implemented #202
  • New data classes and attributes were introduced to orm.py #209
  • The documentation page was updated
  • Unit tests were created #207
  • A CI pipeline was introduced #208
  • The metadata was updated #219

[v0.10.0] Unreleased - Refactoring - 2020-10-08

A complete refactoring took place! Downloading data was entirely changed; introducing layers of code and removing duplicated code while more of less following DRY. Moreover, post-processing was changed to be more accessible and easier to execute. For example, docker now helps to spin up a database container. The documention on RTD was extended, update and improved to be more helpful for new users. Read more about the details:

Added

  • added more technologies
  • added documentation for ReadTheDocs
  • improved parallel download
  • merged all stale branches
  • The class :class:open_mastr.soap_api.mirror.MaStRMirror was introduced for mirroring MaStR data with latest updates #149
  • Introduce project home ~/.open-MaStR/config/ #120
  • Documentation of post-processing #117
  • Updated documentation of downloading data #124 which is harmonized with the other parts of docs and with GitHubs README #135
  • Local execution of post-processing now possible, optionally in dockered database #116
  • Post-processing adapted to CSV data from :class:open_mastr.soap_api.mirror.MaStRMirror #172
  • Tests for changed download code are added #131
  • Metadata added for raw data as frictionless data package #160
  • Suffix columns instead of deferring in database CSV export #157
  • Code examples added for :class:open_mastr.soap_api.mirror.MaStRMirror explaining basic use of mirroring database #164
  • CSV file reader for MaStR raw data added #181
  • Zenodo data upload #173 and the missing LICENSE file for Zenodo is fixed in #186
  • Add postgres database service in CI job for interacting with database in tests #159
  • Tests for :class:open_mastr.soap_api.mirror.MaStRMirror #191
  • Download functionality for Lokationen (with focus on :class:open_mastr.soap_api.mirror.MaStRMirror) #162
  • The CHANGELOG is now included in the documentation

Changed

  • Download of raw data has entirely been refactored. A python wrapper for querying the MaStR API was introduced #83
  • Based on that, for bulk data download, MaStRDownload provides handy query functions for power unit data #86. See also #128
  • configuration through config filen in ~/.open-MaStR/config/ with less hard-coded parameters in source files #120, #112
  • move code into one package named open_mastr #123
  • Switch to GitHub Actions for CI instead of Travis #143
  • Fixed unexpected line breaks during CSV export that corrupted data #170
  • Filtering of duplicates in MaStR data (see MaStR help <https://www.marktstammdatenregister.de/MaStRHilfe/subpages/statistik.html>_) got changed to filter units by leading three characters and select only directly entered data #180
  • Generalize CSV reading function #188

Removed

  • Most of prior code for downloading data

[v0.9.0] Initial Release - Try and Errors - 2019-12-05

Added

  • docstrings for functions
  • tests
  • setup.py file
  • added update function (based on latest timestamp in powerunits csv)
  • added wind functions
  • only download power units for wind to avoid massive download
  • changed : process units wind ("one-click solution")
  • added loop to retry failed power unit downloads, currently one retry
  • write failed downloads to file

Changed

  • rename import-api soap_api
  • update README with instruction for tests
  • update README with instruction for setup

Removed

  • unused imports
  • obsolete comments

Fixed

  • power unit update
  • filter technologies from power units

[v0.8.0] 2019-09-30

Added

  • README.md
  • CHANGELOG.md
  • CONTRIBUTING.md
  • LICENSE
  • continuous integration with TravisCI (.travis.yml)
  • linting tests and their config files (.pylintrc and .flake8)
  • requirements.txt
  • parallelized download for power units and solar
  • utils.py for utility functions
  • added storage units download
  • added wind permit download
  • ontology folder (#46)

Changed