GlitchTip 3.0: Primary key migration
Major version update
GlitchTip's new release introduces performance optimizations and a significant database migration. It marks a major version update (v3.0.0) due to the potentially disruptive migration. If you have a smaller GlitchTip instance with less than 1 million events stored in your database and a few minutes of downtime will not disrupt your workflow, you can upgrade immediately. However, if you are maintaining a larger instance with more than 1 million events, please read on for suggestions about how to reduce downtime.
Primary key migration
This release includes a potentially disruptive change to the app's primary keys. Though we try our best to avoid changes that might interrupt our app's services, GlitchTip needs to migrate its primary keys from INT to BIGINT. With the very high throughput expected in larger GlitchTip instances, it is possible to hit the limits of INT keys. To avoid bigger problems down the line as instances grow, we are taking this step now.
This migration may take a significant amount of time for larger instances and will likely result in downtime unless mitigating steps are taken. We have updated the GlitchTip helm chart to allow the option of manual migration instead of automatically migrating pre-install.
Mitigating migration issues
It is possible to reduce the amount of time needed to migrate larger instances. If tolerable, we recommend reducing the amount of events in your database. The default is to keep events for 90 days. To lower it, set the GLITCHTIP_MAX_EVENT_LIFE_DAYS environment variable to a low number, such as 7. Then wait one day for the scheduled task to clear out the older events. You can confirm the event count was lowered by inspecting the events_event table in PostgreSQL or running a count in a Django shell (./manage.py shell_plus) with Event.objects.count(). Once reduced, run the migration manually using ./manage.py migrate. If using docker, this command needs to run from either a web or worker container/pod. After the migration, you may want to adjust or remove the GLITCHTIP_MAX_EVENT_LIFE_DAYS environment variable.
As active programmers, we at GlitchTip know that disruptive updates can be a source of frustration. But in this case we did not feel we had any other option. We hope you will at least be glad to know we are developing GlitchTip for the long haul.
Performance optimizations
Aside from migrating to BIGINT, this release also includes preliminary work on our ongoing project of boosting GlitchTip's performance, especially in its handling of events.
Support GlitchTip
Your time and financial donations are the only thing that keeps GlitchTip development going. GlitchTip is not backed by venture capital.