The issue is most likely with GTK smooth scrolling. It likely doesn't honor the xmodmap button mappings, as that's sort of a hack that has worked because of the poor scrolling implementation in X. I don't know what the proper resolution is, but I'm pretty certain it's a bug that needs to be resolved in GTK somehow, or marked as "won't fix" by them.
The issue is most likely with GTK smooth scrolling. It likely doesn't honor the xmodmap button mappings, as that's sort of a hack that has worked because of the poor scrolling implementation in X. I don't know what the proper resolution is, but I'm pretty certain it's a bug that needs to be resolved in GTK somehow, or marked as "won't fix" by them.