Using Rawr to Evaluate Raiders' DPS

Topics: Rawr.Addon, Rawr.Base, Rawr.Base.BossHandler, Rawr.Base.Buffs, Rawr.Base.ItemFilters, Rawr.Base.Items, Rawr.Base.Optimizer, Rawr.Bear, Rawr.Cat, Rawr.DPSDK, Rawr.DPSWarr, Rawr.Elemental, Rawr.Enhance, Rawr.Healadin, Rawr.HealPriest, Rawr.Hunter, Rawr.Mage, Rawr.Moonkin, Rawr.ProtPaladin, Rawr.ProtWarr, Rawr.RestoSham, Rawr.Retribution, Rawr.Rogue, Rawr.ShadowPriest, Rawr.TankDK, Rawr.Tree, Rawr.Warlock
Dec 31, 2009 at 2:57 PM

I'm kicking around some ideas about what is a fair way to evaluate my raiders' DPS and want to solicit some advice and comments on the following model.

My current plan is to use Rawr to pull the armory data from each of my raiders to get a rough number of what their DPS should be fully raid buffed.  I'll adjust buffs based on what we have / do not have (for example, we usually don't have a boomkin).  I'll communicate what buffs and debuffs can and cannot be used (and exclude heroism all-together) and ask that all raiders exclude use of their CDs (Avenging Wrath, Mirror Images, Recklessness, etc).

Once I have a full report on each individual DPSer and a rough number of what they should be doing we're heading to Patchwerk,  There, we'll kill him and I'll pull stats from the encounter and cross-reference with the Rawr reports to find the largest deltas between the actual DPS numbers and what Rawr reports they should be doing.

Does this sound reasonable?  Any recommendations on ways to limit problems with this model?

 

 

Dec 31, 2009 at 3:50 PM

This main issue with doing what you are proposing is that on any given fight random numbers play a large role, especially in such a short fight like patchwerk. 

A player may get very lucky on a 2 minute fight with a high crit rate and yet be very sloppy with his rotation, on the other hand somebody else in the raid might nail their dps rotation and just have very bad luck getting crits.

A much better way to evaluate DPS is to look at rotations.  After a raid you can go over a log file and look at the order abilities and cooldowns were used. Some caster classes benefit from standing closer to a boss because of travel time on debuff/ buff abilities.  For other caster classes distance does not matter.  As another example dps meters do not show if a player has issues with 'clipping' their DoTs.  Looking at the logs can help sort these types of issues.

Many DPS classes have either a pure rotation system or a priority on which abilities to use.  What separates a decent player from a good dps player is many times the ability to change targets and continue to follow a rotation or priority order.

Bottom line in my view is that a patchwerk type fight is just a poor way to evaluate dps unless you are looking at it from the standpoint of how good or tight was the players rotation/ spell priority usage.

 

Dec 31, 2009 at 4:07 PM

Yeah, I see your point and it's a good one.

Patchwerk would probably go down in 90 seconds or less at our current gear levels.  That doesn't give a lot of time for things to "even" out.

I've done a lot of evaluation on some folks' rotations but damn that takes a long time (research for me as I don't know every class/spec rotation and sifting through the log files).

 

 

Dec 31, 2009 at 4:11 PM

I wonder if the target dummy on a 7-minute nuke fest is another option?

It allows me to analyze everyone's rotation as their are no mechanic encounters to consider and should allow everyone to stick to their rotations on a very tight basis.

The gotcha is the dot / tick on target dummies.  Don't dots tick for 1 or something like that?

 

Coordinator
Dec 31, 2009 at 6:34 PM

The problem with using Rawr to know how much DPS people should be doing is that not all Rawr models are created equal. Some are very good, and predict potential DPS with a very high degree of accuracy (Mage, Cat, SPriest, DPSWarr, etc). Others consistently overestimate DPS. Some are wildly in accurate in many cases. I think you'd have to use a combination of Rawr, SimCraft, and the leading spreadsheet for several classes/specs.

Dec 31, 2009 at 6:52 PM

Thanks!  That's good to know and I started to notice that as I was pulling up my raiders and noticed that some modules show what the expected DPS should be and some don't offer the details quite as readily.

 

Developer
Dec 31, 2009 at 8:18 PM
Edited Dec 31, 2009 at 8:20 PM

To be honest this is a fools errand. Rawr bases its dps on largely patchwerk style fights. Its not designed to measure dps between classes but instead to advice players how best to improve their gear which is completely different issue. See the other post on this subject also posted today. In that post I gave a detailed reply of what sort of program I thought would be good. ie: one that analysed the logs of a fight and made recommendations as to where improvements could be made. This would pick up slacking dps, low uptimes/uses of abilities as well as missing buffs/debuffs etc.

Dec 31, 2009 at 9:02 PM

I think I understand the caveats you mentioned and of course, consider those things while evaluating raiders.  What I'm trying to avoid is a subjective measurement of one's performance and the ability to provide some type of method or tool that I can use to say to the raider (who I already know is doing poorly) "I'm not sure your dps is where it should be."

Obviously, as previously mentioned I do understand that rotations change based on the mechanics of the encounter.  But also consider alternatives such as this:  I have a mage in that consistently does around 5k dps on even the easiest tank-and-spank encounters (such as the first boss of ToC NB).  I know that 5k dps is below what should be done but I also don't know exactly what is reasonable either.  Said mage is wearing all i245 gear or better.

I can understand running away from a RL that screams that "your DPS isn't what my tool says it should be" but it is possibly worse to let the subjective opinion of the RL be the only determining factor in consideration.

My intention isn't so much as to use the tool to find out who is performing poorly (I think I already know that) but to use the tool to give the at least some idea of where they should be on a simple tank and spank encounter.  After all, if they can't keep their debuffs up and handle procs appropriately in those situations they surely aren't going to be doing so in another more complicated fight.

I've spent a fair amount of time doing the manual analysis of combat logs for several people but frankly, it just takes a long time and I was hoping to try to avoid that very long process for 20+ DPS raiders who regularly raid with us on either priority slot or standby.

Anyway.. thanks again for the advice and recommendations all.  To the combat logs I go!

Dec 31, 2009 at 9:07 PM
SebzCP wrote:

I agree with Levva on this one. One good place to start on improving your raiding experience is addons. I highly recommend you get RankWatch, which will tell you and those affected if they are using an under-ranked spell/skill. This happens more often than you'd think and can really sink performance. I also highly recommend getting another addon called Raid Buff Status, which scans your party/raid to ensure that everyone is buffed up properly to the extent at which the members present can buff other members. For instance, if you have two paladins in your raid and someone only has one paladin buff on, Raid Buff Status will tell you about it. I might also recommend using something like Utopia, which has an uptime module that will chart when buffs were up or not during the encounters; this will let you see who's slacking.

Yep, I do use both Utopia and RaidBuffStatus today.  Both are very help and I appreciate the recommendations.

I'll check out RackWatch too!

 

 

Developer
Jan 1, 2010 at 2:29 AM
Doxa wrote:
I've spent a fair amount of time doing the manual analysis of combat logs for several people but frankly, it just takes a long time and I was hoping to try to avoid that very long process for 20+ DPS raiders who regularly raid with us on either priority slot or standby.

Anyway.. thanks again for the advice and recommendations all.  To the combat logs I go!

This is why I was suggesting (in the other thread) that the creation of a combat log recommendations tool would be a fantastic idea. ie: a tool that looked at the logs looking for the common things we search for when analysing manually. eg: uptimes of buffs/debuffs, activity times, uses of various abilities etc. So that instead of manually trawling the combat logs and the massive amount of stats you get on the likes of worldoflogs.com it goes off to that site pulls down the stats from specific things and presents a report. The report would be Expected Values vs Observed Values and it would highlight where there were deficiencies or omissions.

That way you'd point the tool at a combat log parse and tell it to analyse the stats and report on problems. This would give you a solid report to post on your guild forum to show - look guys we are missing buff X, Y & Z and are only having 75% uptime on A & B when it should be closer to 100%. Oh and you mage there why are you doing an average of X damage per hit of Arcane Blast it should be at least Y given your gear did you forget to train the top rank?

Please note that whilst this is very interesting and would be a fascinating project this is straying well beyond what the purpose of this discussion board is for. I cannot see this being a valid feature to add to Rawr. I can see it as being a really interesting and viable stand alone project. Perhaps a thread at http://elitistjerks.com/f31/ ie: Elitist Jerks Class Mechanics forum would be the place to propose such a tool and to build a team capable of creating it.

Jan 1, 2010 at 5:04 AM
SebzCP wrote:

You'd be much better off trying to post in the official forums...

Okay.  ^ was over the line.   Let's not get crazy here.  ;-)

 

Jan 1, 2010 at 5:47 PM

Just a heads up that since patch 3.3, there are quite a few procs that are no longer getting reported in the combat log via SPELL_AURA_APPLIED messages.

-> http://forums.worldofwarcraft.com/thread.html?topicId=21726253070&sid=1

Long story short - you will not get accurate uptime info for the missing procs until Blizz gets around to fixing it :/

 

On a side note - you can simulate your entire raid via SimCraft :)

 

Developer
Jan 2, 2010 at 12:25 PM
SebzCP wrote:

All they will do is delete anything he posts and ban him.

You'd be much better off trying to post in the official forums, some community forums, or posting a help wanted on sourceforge.net.

What makes you think that? If it is a sensible post looking to create an exceptionally useful application and build a team to do so then its precisely the sort of thing - analysing game stats - that the EJ forum specialises in. If he's worried about getting banned - which I REALLY don't think would happen - then I'd happily start the thread myself.

Jan 2, 2010 at 2:06 PM
SebzCP wrote:

I agree with Levva on this one. One good place to start on improving your raiding experience is addons. I highly recommend you get RankWatch, which will tell you and those affected if they are using an under-ranked spell/skill. This happens more often than you'd think and can really sink performance.

So downloaded this the day you recommended and let me just say I had no idea how often this happens.  Out of the 2-3 heroics I PUG every day it's almost a guarentee that I'll find at least 1 person in the group using down-ranked spells.  Yesterday out of the 5 people in my heroic FoS group only one person wasn't using down-ranked spells.  Rank 5 rejuv really sucks. :-)

Jan 4, 2010 at 7:19 AM
Doxa wrote:

I've spent a fair amount of time doing the manual analysis of combat logs for several people but frankly, it just takes a long time and I was hoping to try to avoid that very long process for 20+ DPS raiders who regularly raid with us on either priority slot or standby. 

Manual analysis of combat logs, both yours and from other guilds is the only way to see what given player do wrong.

Yes, if you have to do it for 20-30 people (don't forget healers and tanks!) it will be very boring and time consuming. But you can assing a few good players as "role advisors" (for example melee dps, range dps, tanks, healers - just to clearly divide responsibility). Teach the few how to filter and read combat logs, and ask them to help the poor ones :)

 

Jan 4, 2010 at 8:39 PM
Edited Jan 4, 2010 at 8:44 PM

There have already a lot of good points been made. I second that you shouldn't be using Rawr (alone) to determine raider performance. I know your wish for an easy way to check performance from my own experience. Checking every player on your own is almost impossible especially since even a great raid leader will hardly know all ~20 viable raid specs good enough to judge a player.

I've been using Simulationcraft to generate a report for the actual raid (including buffs etc) and compare it to a WoL log. However do not compare direct DPS numbers but the difference between theoretical and real dps. For example MageA has a theoretical DPS of 7500 and did 6000 on a particular fight (80%) and MageB had a theoretical DPS of 9000 and did 6200 (69%). If this pattern stays the same over several comparable fights, it's clear that MageB, even though his total DPS is better than MageA, isn't playing to his fullest potential (or there were external factors like lag, low fps etc) and a more detailled check of the combat log would be required. This saves a bit of time.
Also setting up a certain "specialists" council of some kind (class leader, role leader, whatever) takes quite a lot work off of you. Those should of course be very familiar with the according classes and be top players in your raid (they don't neccessary need to play the class as their main, so far I've seen main chars being outperformed by an extremely well played twink). Ask them to look at under-performers and give advice. Here might Rawr be a good help since you can compare differences between members of a class and don't having to worry about different degrees of accuracy from different Rawr modules.
Another way which was kinda successful in our guild (to my big surprise, I expected more trouble) was that the raid leader used a few days to look at every raid char in the guild and made a public post in our forum about his views on the person's performance with possible improvements and the request for comments from others and the player himself. This yielded some very good results and improved the general raid performance. However you need to be very careful with your wording because people may take offense very fast on public criticism.

Another thing I've noticed: Don't only compare DPS but also check for target switches. A target switch almost always costs DPS however is very important to certain boss fights. We have a few people in our raid who constantly are very high on the dps meters however hardly switch targets but stay on the main target if possible at all times to increase their dps (a fully debuffed target provides of course better dps than a newly spawned add). So while they look good on damage meters they are hindering the raid on encounters like Morrowgar or Deathbringer Saurfang. On normal mode that's usually not that important, but I expect hard modes will require fast switches. You can check who did how much damage on which adds by selecting the Creature in WoL and selecting Damage by Actor. Faster/better switches often improve raid performance much more than 200 dps.

Jan 4, 2010 at 10:53 PM
Just wanted to throw my 2cents in here.

Another easy thing you can do is look at percentages of overall damage to what a tool such as Rawr would give (assuming that the Rawr model is current). Many players don't hit the optimal DPS due to lag, movement, etc, but you can tell if their rotation is correct by comparing percentages. Rawr says you should be 20% white swings, 10% SpecialA, 15% SpecialB. The parse shows 30% white swings, 10% SpecialA, and 5% SpecialB. Clearly, something is wrong

You can also look at the GCD usage (The fight should be giving ~200GCDs over a 5min fight. 180 GCDs used is within 10% and is probably doing pretty well; 120 GCDs, this guy did something wrong.

Also, many online logging tools will show procs. You can look at how many Overpowers were lost by counting Taste for Blood procs and target dodges, and comparing to the number of Overpowers cast. Things like that.

Unfortunately, these are all model specific, so it would need the expertise of people who know how to play the spec correctly. It's definitely an interesting topic, and one we've discussed in the past. I'm sure you could find some people willing to help you, even if you only start out parsing one or two specs. You gotta start somewhere...didn't Rawr start off as Bear-only? :P