To see all of the albums within a gallery, move the mouse to the icon at the bottom left of the album viewer.