For new divers, I would recommend getting a mask, snorkel and fins with your open water certification. The next items to buy are exposure protection and a computer. The exposure protection is prioritized as rental equipment is often difficult to fit. The computer is prioritized as knowing the menus and having your own decompression algorithm is important for safe diving.
It typically makes sense to rent BCD, regulator, tank and weight.
As you progress, your diving may necessitate a dive light or cutting implement.
Finally for boat diving, it is a good idea to have a surface signaling device.
For dive trips, putting together an inexpensive save a dive kit can avoid quite a bit of frustration.
Mask/defog - Buy
Snorkel - Buy
Fins/Booties - Buy
Computer/Gauges/Compass - Get a wrist computer
BCD - Rent
Regulator - Rent
Tank - Rent
Weight - Rent
Wetsuit - Buy
Hood - Own for cold weather
Gloves - Own for cold weather
Dive Light - Depends on diving
Knife - Depends on diving
Log Book - Keep up to date
Cert Card - Know where it is
Surface Signaling Device - For boat diving
Save-A-Dive Kit - As needed
Some thoughts on training. I started with Open Water and took a couple of classes that were interesting to me. Peak Performance Buoyancy and Navigation. I would like to take Stress and Rescue, Deep Dive and Search and Recovery, but am not quite certain how I want to go about it. I currently have no intention of becoming a Dive Master or Instructor, but I would like to introduce my daughters to Scuba Diving.
SSI Stress and Rescue - $265 (A-1 Scuba)
Deep Dive - Homestead or Blue Hole - $165 + $70 Digital Online Materials
Search and Recovery - Homestead, Blue Hole, Chatfield
Night & Limited Visibility - I have the minimum (5) dives