I disagree. Gaining an additional 15-30 seconds on the line to discuss strategies does impact the game. And stepping of the line early to move backwards makes it easier for the handlers to accurately read the pull (especially if it is about to land in the endzone).
Additionally, both teams should follow the time limits. And I might be mistaken but non-players still on the pitch are considered a pre-pull timing violation and not offside, according to the Appendix A5.4.1.4.
But I agree maybe the annotation could be more explicit.