South Africa’s biggest forum. Discuss, discover, and connect with thousands of members.
What do you guys use? Considering Hadoop on HDFS, but the single-point-of-failure thing is a problem. How would you go about getting around that?
Where I work ... we use a simpler approach.
Heartbeat to manage the high availability.
For the database, Mysql replication. We use the Master - Master configuration.
HTH
Looking at a few petabytes of data. Is mysql going to be feasible for that?
The databases average about 20Gigs. You saying you looking at petabytes ... goodluck with the backups for that.
I cant help / comment anymore, as I have no experience with management and handling of that amount of data. Best of luck.