Skip to content

[5pt] Run-time improvements for fimpact processes #1864

Description

@RyanSpies-NOAA

Noted significant fim_pipeline.sh runtime for HUC 07120004 (Chicago metro nr O'Hare airport). Looking at the UAT run time history (total runtime):
4_9_14_0 --> 152 mins
4_9_12_1 --> 155 mins
4_9_11_0 --> 78 mins
4_9_10_10 --> 28 mins
4_9_8_0 --> 31 mins

I added runtime counters for the process_roads_fimpact.py and process_buildings_fimpact.py and flagged some significant runtime for the buildings process. This was evident in branch 0 and in some of the GMS branches:

Image Image

Looking through the code, it appears possible bottlenecks are the polygon-on-polygon intersections (geopandas) and the zonal_stats calls.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

refactoringRefacting code to obtain the same result

Type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions