Every website you visit installs trackers onto your computer and slows it down subtly. These are called cookies and cached data, which work to track your activity if you regularly visit the same site.
YouTube could slow down sometimes without you knowing because of the website and app caches. If this is enough to bother you, then you might want to clear them on your phone or desktop right away.