Testing Wave #3
Over the last four weeks I have conducted three testing sessions involving my four Unreal Tournament multiplayer maps; CTF-Void, CTF-Redemption, DM-Midnight, and DM-Peon. I averaged at around 10 testers per session, and they were able to test each map multiple times, including with changes.
Each week I asked them to fill out a form, asking them what they would rate specific attributes of the maps (Gameplay, Navigation, Weapon Placement / Use, and Map Size). Over the four weeks I was able to receive over 35 replies.
This blog, as well as detailing this quantitative data, will discuss and reflect on my map designs to their creation and testing. After which I will be choosing one map to further develop and test in the future with the inclusion of certain testing tools / skills such as heat-maps and the use of meshes to flesh out the level.
I noticed that most of my maps had difficulty complimenting Unreal Tournament 4's gameplay. As I designed and developed the blockouts without sufficient research / playtesting, the design were flawed before I had even started. The differences between Unreal Tournament 3 and Unreal Tournament, though subtle at times, change certain gameplay elements drastically, meaning the maps I designed were more built for UT3's gameplay. As already explained in an earlier post, I favour close-quarters combat over long-range skirmishes, or open-planned maps. By testing, it has made it abundantly clear that most people have their own ideas of how they want to play the map and their own methods; it's improbable to suit them all.
Digressing back to the testing, as I will make a larger reflection post later in the semester, out of all the testing and feedback I am confident which map I am going to focus on, but first, the data.
The first question on the form is 'what would you rate the gameplay of this map out of 5?'. This covers how fun the map was to play.
Redemption scored the lowest at an average of 2/5 with a slight curve dropping from 2-4 respectively.
Out of both of the CTF maps Void averaged a higher score with an average of 3, with the addition of quite a few 4/5's and even a single 5/5 score. It follows quite an average curve with 3 being the highest point.
Both Deathmatch maps scored greater than the ctf maps for their gameplay. Starting with DM-Peon: at an average score of 3, with 4 being the second greatest number, comes quite close to CTF-Void's scoring, the difference is DM-Peon doesn't have a score of 1, and boasts far more 5/5's than Void
DM-Midnight, however, trumps all of the maps with an average of 4/5 and a considerable amount of 5/5's making it the highest scored map for its gameplay.
The second question in the form was 'what would you rate the navigation of the map?' this covers the map flow, pickups and accessibility of the map.
Again starting with ctf-redemption, scoring the lowest again at an average of 3 with a steep curve from 1-3 with a single score for both 4/5 and 5/5.
ctf-void has a rather simple curve with its metrics; with an average of 3/5 it curves with similar scores between 1/5 & 5/5 and 2/5 & 4/5.
dm-peon has a lot easier chart to read as it looks like a generic 1st, 2nd, and 3rd place podium with 3/5 as the average score for its navigation.
dm-midnight again averages at 4/5 with a considerable amount of 5/5's as a sharp curve over the chart.
The third question in the form was 'what would you rate the map size?' this covers distances to the flags in ctf, and how easy it is to traverse the map to find and engage in combat.
ctf-redemption averages at 2/5 for its map size with quite a few 1/5's and 3/5's to balance it out.
ctf-void follows a similar curve to its other charts and averages at 3/5 with a few 2/5's and even fewer 4/5's and 1/5's
dm-peon just barely scrapes an average of 3/5 for its map size, with a very close amount of 2/5's and the rest being 4/5's.
dm-midnight is very close between the most amount of votes for 4/5 and 5/5. The considerable amount of 3/5's put the average at 4/5
Weapon Placements / Use
The fourth question in the form was 'what would you rate the weapon placements / use?' this covers the placement of weapons, and which weapons are used within the level; do they compliment the map?
ctf-redemption has a simple curve with a steep change between 1/5 and 2/5, with 2/5 as the average with quite a few 3/5 scores.
ctf-void follows suit with yet another simple curve with a steady increase between 1/5 - 3/5
dm-peon's chart is very similar to its navigation score, difference is, for the weapons it scores much more 4/5's.
dm-midnight has a strong 4/5 average with only a few amount of votes for 2/5 and 3/5
Now that I have presented the metrics of the form's data, I will now cover the analytics of the information given. This section will detail why I think the scores were given to each map for each section, and compare them.
Each of the maps were rated out of 5 for their gameplay; how fun was the map? Overall dm-midnight was the highest rated with an average of 4-5/5, whereas, ctf-void and dm-peon averaged a 3/5, and ctf-redemption averaging 2/5.
I see this due to how the maps were designed. CTF-void was designed using inspiration from ctf-facingworlds, a simple two base, open-planned map with long to short range skirmishes. Although fun, it still had the problem of players constantly falling off the map. Where most people testing ctf-void enjoyed this feature as they thought it was riskier and more rewarding, other players were constantly annoyed at their progress being stunted by a simple shock rifle hit.
CTF-redemption is the complete opposite of ctf-void and was rated very low, I see this because the map was too small, despite the designs and vision in my head the map being much larger. When blocked out, the map was very claustrophobic and tight, and although it did gain a few generous votes, it was still clear that this map was weak and not that fun. I personally believe that the flow was disrupted by the bridge and ramp in the middle, there was no other incentive to traverse the side routes, as this route was a lot quicker and provided enough cover for the flag runner to return the enemy's flag easily.
Comparing the ctf maps to the deathmatch maps it is clear that I did a better job designing deathmatch maps and executing them. DM-midnight did amazingly well and received critical acclaim from peers and testers. DM-midnight is a small close-quarters map, whereas dm-peon was designed more around open-scaled Halo maps. The problem here is that Unreal Tournament 4 relies a lot of Z-fights (multiple floors rather than open-planned), and dm-peon was just too open, lots to run around on and seemed quite flat. Surprisingly, it still did well, however, the main complaints were running around finding people, and not a lot to maneuver on due to the flat design. DM-midnight is designed a lot more around a Unreal Tournament 3 style map, but at the time I didn’t realise how different UT3 was to UT4.
It seems that ctf-redemption relied too heavily on ignoring the weapons completely and running straight forward, whereas a defender had easy access to a rocket launcher, making it debatably easy to ward off attackers due to the limited small access points to the map. CTF-void also includes rocket launchers, but due to the open-scale of the map it was easy to use the lock-on feature, adding more ways for a player to be sent off the side of the map. DM-midnight also had issues with the rocket launcher, as it was very versatile wherever the player was on the map, removing it created a much better flow, and made them fight for the armour in the middle creating a constant flow.
Due to the metrics of the testing over the last few weeks, I feel like I can make an informed decision of what one map I can focus on to further develop. So I believe, due to the data received, that I should focus on working on dm-midnight. Next week and thereon I am doing to focus on testing this map and provide more level design testing skills such as heat maps, etc.