I am sure you have heard the familiar adage espoused by most database and storage vendors - Stripe And Mirror Everything (SAME). This theory is particularly pushed by Oracle, after all databases are the most important assets that occupy your storage arrays. While we all understand that the SAME methodology while trying to maintain simplicity has performance as its main goal. And I will be the first person to say that far too often I/O performance issues on databases are primarily because of bad storage design and the use of concat (non-striped) files for the database. These days with the bundling of many storage functions within the database (Oracle), and with the use of thin provisioning or storage virtualization most people assume storage being such a mature technology will be bullet proof when it comes to performance and availability. And that is where I would caution people especially DBAs, to pay attention to where their database files live.
Today most storage arrays employ some form of thin provisioning. To get to thin provisioning there are multiple abstractions that need to happen behind the scenes on the storage array. To put it simply, most arrays these days start of with some variation of RAID5 or RAID6. Each such RAID group is drawn from a set of multiple disks like 3D+1P or 7D+1P or 14D+2P (D means Disk that stores actual user Data and P is the disk dedicated to store the RAID parity). So availability is baked in to some extent in both RAID5 and RAID6. RAID5 can withstand a single disk failure within its RAID group and RAID6 can withstand 2 disk failures. Now, historically RAID5 and RAID6 have suffered performance penalty and bias simply because of the extra overhead to calculate parity compared to RAID10 which is simply a stripe & mirror approach. While the performance penalty has come down to some extent, they still exist. To compensate for some of the performance limitation and to manage huge amounts of data that needs to be provisioned, what most storage arrays do is to combine multiple RAID groups into a pool, picture this as a pool of RAID groups with each group having multiple disks as explained above. They then create multiple such pools. Now, in each pool they carve out slices of disks that stripe across all RAID groups in that pool and hence across all disks in each RAID group, for simplicity lets call these logical devices (LDEV). Now, they combine multiple LDEVS across all pools and create the actual LUN seen by the host. The host then can stripe across these LUNs again using volume managers like Veritas, LVM etc. Tired, me too... just look at the amount of striping and aggregation that is happening. So now when you create a database file on one of the LUNs, you are in effect putting small chunks of that file across all possible disks on that array. Your neighbor on another host will do the same thing since the storage array being so big is shared across multiple servers. So as far as performance you may think you have hit it out of the ball park but just imagine this - in the thousands of disk in that array, if 2 disks fail in a RAID5 group or 3 in a RAID6 group, then you will bring down every application and database that is connected to that storage array. You can say how often can that happen, what are the chances of 2 or 3 disks failing in an single RAID group, it has happened before and it will happen again. Your goal should not be to take a chance with whatever low probability an event affords but assume it can happen and take a little precaution. Another issue with this share and stripe philosophy is the seemingly lack of control you will have to prevent other I/O intensive applications hurting your application's performance. This is why I tell people that if you can stripe 8 wide and can segregate your database to its own set of disks then you will have the performance you need and the peace of mind. You may waste some space, but what is the cost today of some storage waste when you can buy 1TB for $50 at Costco! Striping beyond 16 disks on a database has seldom seen any positive effect on performance, and containing your space ensures you are not exposed to the vagaries of other applications and prevent exposure to risk of a single disk somewhere bringing everything down. So a saner approach would be "stripe but not too far and wide!"
Comments