Hi Anne-Marie,
For most EU citizens it is possible to travel to Serbia without a proper passport (with an ID only). For UK citizens that actually means taking a proper passport. A visa is required only if you stay longer than 30 or 60 days. You can always try to find that information at your Ministry of Foreign Affairs website. They usually post all general information with regard to travelling and required documents.
Transporting your camera equipment across the borders should not be a problem at all. Though I have no idea whether you would be able to rent a car in one country and return it in another. Taking in consideration that the car would leave EU territory I would find that even more unlikely. But, you have made me curious and I checked two or three sites. Some search engines allow Belgrade as a return location others are limited to one country only. I believe the best idea would be to speak to one of them directly or have a Romanian fixer check that for you to be on a safe side.
Good luck with your shoot.
Kind regards,
Tomek Hryniszyn
BALTICA FILM