SQLAlchemy index error on Pulling data from Google Analytics

I'm using a python script to fetch data from Google Analytics and i'm getting the below error. Can someone please help me to understand what this error is and suggest something to resolve this error ?

05:04:31 sqlalchemy.exc.OperationalError: (psycopg2.errors.ProgramLimitExceeded) index row size 2792 exceeds btree version 4 maximum 2704 for index "ga_events_pkey"
05:04:31 DETAIL:  Index row references tuple (42550,53) in relation "ga_events".
05:04:31 HINT:  Values larger than 1/3 of a buffer page cannot be indexed.
05:04:31 Consider a function index of an MD5 hash of the value, or use full text indexing.

Below is the current Primary Key What change i need to do on this Primary Key, Seen lot of posts to create an index based on md5, But i'm confused whether i need to remove the below index and recreate index ? Or create a new one and md5 on all columns ?

Kindly guide.

ALTER TABLE dl_ga.ga_events
    ADD CONSTRAINT ga_events_pkey PRIMARY KEY ("ga:date", "ga:eventCategory", "ga:eventAction", "ga:eventLabel", "ga:source", "ga:medium", "ga:campaign", view_id);
    

Tried to modify the index like below, But getting error,May i know how this can be created to avoid this error?

ALTER TABLE dl_ga.ga_events ADD CONSTRAINT ga_events_pkey PRIMARY KEY( 
md5("ga:date"), md5("ga:eventCategory"), md5("ga:eventAction")
, md5("ga:eventLabel"), md5("ga:source"), md5("ga:medium")
, md5("ga:campaign"), md5(view_id)
);

Code where exception is throwing:

if data:
   push_data_to_pg(pg_client, ga_load_config.dl_schema,
   f"{ga_load_config.table_prefix}_{ga_load_config.report_name}", data,primary_key_columns=dimensions)

Sample insert query:

    [SQL: INSERT INTO dl_ga.ga_events ("ga:date", "ga:eventCategory", "ga:eventAction", "ga:eventLabel", "ga:source", "ga:medium", "ga:campaign", "ga:uniqueEvents", "ga:totalEvents", view_id) VALUES (%(ga:date_m0)s, %(ga:eventCategory_m0)s, %(ga:eventAction_m0)s, %(ga:eventLabel_m0)s, %(ga:source_m0)s, %(ga:medium_m0)s, %(ga:campaign_m0)s, %(ga:uniqueEvents_m0)s
, %(ga:totalEvents_m0)s, %(view_id_m0)s), (%(ga:date_m1)s, %(ga:eventCategory_m1)s, %(ga:eventAction_m1)s, %(ga:eventLabel_m1)s, %(ga:source_m1)s, %(ga:medium_m1)s, %(ga:campaign_m1)s, %(ga:uniqueEvents_m1)s, %(ga:totalEvents_m1)s, %(view_id_m1)s), (%(ga:date_m2)s, %(ga:eventCategory_m2)s, %(ga:eventAction_m2)s, %(ga:eventLabel_m2)s, %(ga:source_m2)s, %(ga:medium_m2)s, %(ga:campaign_m2)s, %(ga:uniqueEvents_m2)s
, %(ga:totalEvents_m2)s, %(view_id_m2)s), (%(ga:date_m3)s, %(ga:eventCategory_m3)s, %(ga:eventAction_m3)s, %(ga:eventLabel_m3)s, %(ga:source_m3)s, %(ga:medium_m3)s, %(ga:campaign_m3)s, %(ga:uniqueEvents_m3)s, %(ga:totalEvents_m3)s, %(view_id_m3)s), (%(ga:date_m4)s, %(ga:eventCategory_m4)s, %(ga:eventAction_m4)s, %(ga:eventLabel_m4)s, %(ga:source_m4)s, %(ga:medium_m4)s, %(ga:campaign_m4)s, %(ga:uniqueEvents_m4)s, %(ga:totalEvents_m4)s, %(view_id_m4)s), (%(ga:date_m5)s, %(ga:eventCategory_m5)s, %(ga:eventAction_m5)s, %(ga:eventLabel_m5)s, %(ga:source_m5)s, %(ga:medium_m5)s, %(ga:campaign_m5)s, %(ga:uniqueEvents_m5)s, %(ga:totalEvents_m5)s, %(view_id_m5)s)
, (%(ga:date_m6)s, %(ga:eventCategory_m6)s, %(ga:eventAction_m6)s, %(ga:eventLabel_m6)s, %(ga:source_m6)s, %(ga:medium_m6)s, %(ga:campaign_m6)s, %(ga:uniqueEvents_m6)s, %(ga:totalEvents_m6)s, %(view_id_m6)s), (%(ga:date_m7)s, %(ga:eventCategory_m7)s, %(ga:eventAction_m7)s, %(ga:eventLabel_m7)s, %(ga:source_m7)s, %(ga:medium_m7)s, %(ga:campaign_m7)s, %(ga:uniqueEvents_m7)s, %(ga:totalEvents_m7)s, %(view_id_m7)s), (%(ga:date_m8)s, %(ga:eventCategory_m8)s, %(ga:eventAction_m8)s, %(ga:eventLabel_m8)s, %(ga:source_m8)s, %(ga:medium_m8)s
, %(ga:campaign_m8)s, %(ga:uniqueEvents_m8)s, %(ga:totalEvents_m8)s, %(view_id_m8)s), (%(ga:date_m9)s, %(ga:eventCategory_m9)s, %(ga:eventAction_m9)s, %(ga:eventLabel_m9)s, %(ga:source_m9)s, %(ga:medium_m9)s, %(ga:campaign_m9)s, %(ga:uniqueEvents_m9)s, %(ga:totalEvents_m9)s, %(view_id_m9)s), (%(ga:date_m10)s, %(ga:eventCategory_m10)s, %(ga:eventAction_m10)s, %(ga:eventLabel_m10)s, %(ga:source_m10)s, %(ga:medium_m10)s, %(ga:campaign_m10)s, %(ga:uniqueEvents_m10)s, %(ga:totalEvents_m10)s, %(view_id_m10)s), (%(ga:date_m11)s, %(ga:eventCategory_m11)s, %(ga:eventAction_m11)s, %(ga:eventLabel_m11)s, %(ga:source_m11)s, %(ga:medium_m11)s, %(ga:campaign_m11)s, %(ga:uniqueEvents_m11)s, %(ga:totalEvents_m11)s, %(view_id_m11)s), (%(ga:date_m12)s, %(ga:eventCategory_m12)s, %(ga:eventAction_m12)s, %(ga:eventLabel_m12)s, %(ga:source_m12)s, %(ga:medium_m12)s, %(ga:campaign_m12)s, %(ga:uniqueEvents_m12)s, %(ga:totalEvents_m12)s, %(view_id_m12)s), (%(ga:date_m13)s, %(ga:eventCategory_m13)s, %(ga:eventAction_m13)s, %(ga:eventLabel_m13)s, %(ga:source_m13)s, %(ga:medium_m13)s
, %(ga:campaign_m13)s, %(ga:uniqueEvents_m13)s, %(ga:totalEvents_m13)s, %(view_id_m13)s), (%(ga:date_m14)s, %(ga:eventCategory_m14)s, %(ga:eventAction_m14)s, %(ga:eventLabel_m14)s, %(ga:source_m14)s, %(ga:medium_m14)s, %(ga:campaign_m14)s, %(ga:uniqueEvents_m14)s, %(ga:totalEvents_m14)s, %(view_id_m14)s), (%(ga:date_m15)s, %(ga:eventCategory_m15)s, %(ga:eventAction_m15)s, %(ga:eventLabel_m15)s, %(ga:source_m15)s, %(ga:medium_m15)s, %(ga:campaign_m15)s, %(ga:uniqueEvents_m15)s, %(ga:totalEvents_m15)s, %(view_id_m15)s), (%(ga:date_m16)s, %(ga:eventCategory_m16)s, %(ga:eventAction_m16)s, %(ga:eventLabel_m16)s, %(ga:source_m16)s, %(ga:medium_m16)s, %(ga:campaign_m16)s, %(ga:uniqueEvents_m16)s, %(ga:totalEvents_m16)s, %(view_id_m16)s), (%(ga:date_m17)s, %(ga:eventCategory_m17)s, %(ga:eventAction_m17)s, %(ga:eventLabel_m17)s, %(ga:source_m17)s, %(ga:medium_m17)s, %(ga:campaign_m17)s, %(ga:uniqueEvents_m17)s, %(ga:totalEvents_m17)s, %(view_id_m17)s), (%(ga:date_m18)s, %(ga:eventCategory_m18)s, %(ga:eventAction_m18)s, %(ga:eventLabel_m18)s, %(ga:source_m18)s, %(ga:medium_m18)s, %(ga:campaign_m18)s, %(ga:uniqueEvents_m18)s, %(ga:totalEvents_m18)s, %(view_id_m18)s), (%(ga:date_m19)s, %(ga:eventCategory_m19)s, %(ga:eventAction_m19)s, %(ga:eventLabel_m19)s, %(ga:source_m19)s, %(ga:medium_m19)s, %(ga:campaign_m19)s, %(ga:uniqueEvents_m19)s, %(ga:totalEvents_m19)s, %(view_id_m19)s), (%(ga:date_m20)s, %(ga:eventCategory_m20)s, %(ga:eventAction_m20)s, %(ga:eventLabel_m20)s, %(ga:source_m20)s, %(ga:medium_m20)s, %(ga:campaign_m20)s, %(ga:uniqueEvents_m20)s, %(ga:totalEvents_m20)s, %(view_id_m20)s), (%(ga:date_m21)s, %(ga:eventCategory_m21)s, %(ga:eventAction_m21)s, %(ga:eventLabel_m21)s, %(ga:source_m21)s, %(ga:medium_m21)s, %(ga:campaign_m21)s, %(ga:uniqueEvents_m21)s, %(ga:totalEvents_m21)s, %(view_id_m21)s), (%(ga:date_m22)s, %


Read more here: https://stackoverflow.com/questions/64403240/sqlalchemy-index-error-on-pulling-data-from-google-analytics

Content Attribution

This content was originally published by Linu at Recent Questions - Stack Overflow, and is syndicated here via their RSS feed. You can read the original post over there.

%d bloggers like this: