Physics departments are generally small enough that you can get to know everyone on a pretty strong level, and everyone in the department is going to be especially intelligent (at least, the students who will actually be able to make it through the program) and able to have VERY interesting conversations. I agree with everything dolphinrisky, omnieiunium, and Ridley said, but I really want to emphasize to get to know your fellow majors and your professors. I'll be graduating with degrees in Physics and Applied Mathematics this year. You won't learn everything you need in your physics classes, so don't hesitate to branch out. Now I'm working in particle cosmology, and sure enough that year of group theory is proving crucial to my understanding of topological defects and cosmic strings.
I was lucky enough to take a year of group theory because I found it interesting. The more math you take, the easier your physics classes will get. Math is best taught by mathematicians, and while I have no doubt some here will disagree with me, the formality and rigor you learn in math classes can only help you avoid the extremely subtle mistakes that are so easy to make when working on problems in physics. While that's technically true, it's extremely unproductive in the long run. I saw some people mention that your physics classes will probably teach you the math you need. Physics is hard enough on its own if you have trouble with the math, it will only make the physics even harder. The other thing I would focus on is math. You'll find yourself confronted with problems that will take you hours to complete, and having some people you can call up and look to for advice will be very helpful.
These are all great bits of advice, particularly the parts about remembering to be social.