The DVD and VHS videos are located on the 3rd floor, immediately to your left when you enter the doorway. Look for the rows of waist-high shelves (see photo).
CDs and LPs are also located on the third floor in room 302, which is located in the hallway to the north of the elevator foyer, near the lockers.