Pandas add columns and index dynamically and update existing columns

I need to create a dataframe, save it and update it dynamically. timestamp is supposed to be the index. I will get price and qty as column values every minute of the timestamp in a dict. Each time the price would change. If it changes, I have to update the old qty value.

The end result of the dataframe is as follows:

timestamp                100     200     300
2020-11-01 12:00:00       4       3       5

The the next minute I will have data in a list of dict like

[{'timestamp': '2020-11-01 12:01:00', 'price': 200, 'qty': 6},
{'timestamp': '2020-11-01 12:01:00', 'price': 400, 'qty': 3},
{'timestamp': '2020-11-01 12:01:00', 'price': 500, 'qty': 1}]

The dataframe should update like so:

timestamp                100     200     300    400    500
2020-11-01 12:00:00       4       3       5      0      0
2020-11-01 12:01:00       4       9       5      3      1

so basically creating a historical change as well as adding new columns on the fly if new prices are added to the existing dataframe. I will store the dataframe in csv every minute and read and update the new values.

New values coming in the dict in above example; at price 200, the qty is sum of the previous and new value. in addition, if qty is -1, the value should be updated to 0.

Any help on this is appreciated.



Read more here: https://stackoverflow.com/questions/64957100/pandas-add-columns-and-index-dynamically-and-update-existing-columns

Content Attribution

This content was originally published by Pavneet Singh at Recent Questions - Stack Overflow, and is syndicated here via their RSS feed. You can read the original post over there.

%d bloggers like this: