I'll start by saying this is community dominated by 335 drivers. Some consider their cars to be the best thing since sliced bread and will throw low-blows to the 328 and it's "lowly status". Keep that in mind when reading the responses here.
To answer your question directly: it's not slow, but it's not the beast that is the 335. A 328i with 6MT will do 0-60 in 6s flat, even slightly under depending on who you ask. I believe both C&D and Edmunds reviewed it at 5.9s. I think the car is significantly better with the 6MT - the AT seems to sap a bit of life out of the engine (and will probably cost you ~0.5s 0-60).
You need to go drive one for yourself and do your homework first. Any review shows the 328 right around 6s, so start by reading up.
Sport Package (ZSP) includes sport seats, sport steering wheel (slightly thicker than stock), upgraded wheels, and shadowline (black) trim around the windows. If you get a non-Xi, it also gives you the sport suspension. xDrive cars have their own unique suspension setup as it is, so opting for ZSP doesn't change anything.
There are very few (read: pretty much zero) engine modifications for the NA engine. Air filter, intake, and exhaust are certainly do-able, but you won't be getting much more power from the car regardless of what you do. Remember, this is not a car built for straight line speed. The 3 Series is about as balanced as it gets with perfect weight distribution - the smooth power delivery and excellent chassis make the car a blast in the twisties. If you want raw horsepower and muscle, you might want to look elsewhere.