Ingestion Models#
We try to make CDP infrastructure as easy as possible to set up while also allowing for scaling to support as much specific data as can be given and processed.
To this end, our event data ingestion model can be seen below in two examples.
The bare minimum.
A filled out event with event minutes items discussed during the event attached.
The more data provided, the more data displayed and queryable from CDP services.
For more information on each field available, please see the object definition documentation.
Updating Existing Database Documents#
During the event gather pipeline, if an existing database document is found for an ingestion model, the content of the existing database document will be updated if there are differences between the ingestion and database data.
All non-primary fields of the database document will be overwritten with the value of the corresponding fields in the ingestion model.
If the ingestion model field is not provided or None, the database document’s
field will not be overwritten.
Minimal Event Data#
Note: while this looks like a Python Dict, these objects are of type:
IngestionModel.
{'body': {'name': 'Full Council', 'is_active': True},
'sessions': [{'session_datetime': datetime.datetime(2024, 3, 1, 17, 53, 57, 79337),
'video_uri': 'https://video.seattle.gov/media/council/brief_072219_2011957V.mp4',
'session_index': 0}]}
Expanded Event Data with Event Minutes Items#
Note: while this looks like a Python Dict, these objects are of type:
IngestionModel.
{'body': {'name': 'Full Council', 'is_active': True},
'sessions': [{'session_datetime': datetime.datetime(2024, 3, 1, 17, 53, 57, 79345),
'video_uri': 'https://video.seattle.gov/media/council/council_101220_2022077V.mp4',
'session_index': 0},
{'session_datetime': datetime.datetime(2024, 3, 1, 17, 53, 57, 79347),
'video_uri': 'https://video.seattle.gov/media/council/council_113020_2022091V.mp4',
'session_index': 1,
'video_start_time': '00:00:00',
'video_end_time': '99:59:59',
'caption_uri': 'https://www.seattlechannel.org/documents/seattlechannel/closedcaption/2020/council_113020_2022091.vtt'}],
'event_minutes_items': [{'minutes_item': {'name': 'Inf 1656'}},
{'minutes_item': {'name': 'CB 119858'},
'matter': {'name': 'CB 119858',
'matter_type': 'Council Bill',
'title': 'AN ORDINANCE relating to the '
'financing of the West Seattle '
'Bridge',
'result_status': 'Adopted',
'sponsors': [{'name': 'M. Lorena González',
'is_active': True,
'seat': {'name': 'Position '
'9',
'roles': [{'title': 'Council '
'President'},
{'title': 'Chair',
'body': {'name': 'Governance '
'and '
'Education',
'is_active': True}}]}},
{'name': 'Teresa Mosqueda',
'is_active': True,
'picture_uri': 'https://www.seattle.gov/Images/Council/Members/Mosqueda/Mosqueda_225x225.jpg',
'seat': {'name': 'Position '
'8',
'roles': [{'title': 'Chair',
'body': {'name': 'Finance '
'and '
'Housing',
'is_active': True}},
{'title': 'Vice '
'Chair',
'body': {'name': 'Governance '
'and '
'Education',
'is_active': True}}]}}]},
'supporting_files': [{'name': 'Amendment 3',
'uri': 'http://legistar2.granicus.com/seattle/attachments/789a0c9f-dd9c-401b-aaf5-6c67c2a897b0.pdf'}],
'decision': 'Passed',
'votes': [{'person': {'name': 'M. Lorena González',
'is_active': True,
'seat': {'name': 'Position 9',
'roles': [{'title': 'Council '
'President'},
{'title': 'Chair',
'body': {'name': 'Governance '
'and '
'Education',
'is_active': True}}]}},
'decision': 'Approve'},
{'person': {'name': 'Teresa Mosqueda',
'is_active': True,
'seat': {'name': 'Position 8',
'roles': [{'title': 'Chair',
'body': {'name': 'Finance '
'and '
'Housing',
'is_active': True}},
{'title': 'Vice '
'Chair',
'body': {'name': 'Governance '
'and '
'Education',
'is_active': True}}]}},
'decision': 'Approve'},
{'person': {'name': 'Andrew Lewis',
'is_active': True,
'seat': {'name': 'District 7',
'image_uri': 'http://www.seattle.gov/Images/Clerk/district7_50x50.jpg',
'roles': [{'title': 'Vice '
'Chair',
'body': {'name': 'Community '
'Economic '
'Development',
'is_active': True}}]}},
'decision': 'Approve'},
{'person': {'name': 'Alex Pedersen',
'is_active': True,
'seat': {'name': 'District 4',
'roles': [{'title': 'Chair',
'body': {'name': 'Transportation '
'and '
'Utilities',
'is_active': True}}]}},
'decision': 'Reject'}]}]}