Matt Birchler has a design change suggestion for the Workouts app on the Apple Watch:

My proposal is very simple, I just want to be able to dismiss the after-workout summary without scrolling through the whole thing.

One thought was to just move the current “done” button to the top and push everything else down, but that would push the time off screen, which is annoying  as well. That lead me to using the standard system back button used all over watchOS already, and I think it’s a good fit here. You lose zero vertical space for the content and it uses a system control users understand.

I support this suggestion 100%.

Probably nine times out of ten, I forget to scroll all the way down and tap the Done button, and then find it still waiting there for me the next time I open Workouts. The tenth time out of ten, when I do remember to tap the button, nothing happens and I have to tap it a second time.