|
Post by Matt993f.o.d on Oct 11, 2007 22:09:20 GMT
I didnt know where else to start this thread. Please feel free to move it to a more sensible place.
I feel a tiny bit off with the style of review scoring we have adopted here. I think we are including too few factors when grading a sword.
At the moment, we have (correct me if I miss one out) Historical accuracy, structural integrity, handling and value for money.
What about aesthetics, or quality control? I think the inclusion of these two extra categories could vastly improve the way in which we help others understand our own impression on the swords we review.
Finally, our final score is decided by ourselves, but perhaps it would be better if we got the calculator out and actually worked out an average of the scored we give in each area, so as to give a truer representation of the sword. After all, if the sword looks crap, handles crap, and is historically inaccurate, but the reviewer truly loves tough swords, they may give the sword a far far higher score than another person may give it. This can give an unfair impression to people who read the review. I don't know if people do this already, but it did suddenly occur to me that I never did in my reviews.
What do you all think? I'm not trying to force any change here, I would just like to know that people are thinking of these things.
|
|
|
Post by ShooterMike on Oct 11, 2007 22:16:19 GMT
Matt, you aren't the only one thinking about this. I'd like to echo your thoughts. And it's something I've struggled with in writing reviews...
|
|
|
Post by Matt993f.o.d on Oct 11, 2007 22:26:18 GMT
The issue I have is that the categories for scoring that we have are too narrow for me to score the swords that I like for the things that we personally like them for.
However, I think that we must also fall away from the trap of scoring swords too highly simply because of our own personal preferences.
Therefore, I think that the addition of two extra categories, aesthetics and quality control, plus an average of all categories to produce the final score, will make for much more informative reviews, as each reader will be able to choose a sword that matches their own personal tastes, whilst knowing the TRUE overall picture as well.
Do I not hit close to the mark?
|
|
Deleted
Deleted Member
Posts: 0
|
Post by Deleted on Oct 11, 2007 23:08:56 GMT
<snip> Therefore, I think that the addition of two extra categories, aesthetics and quality control, plus an average of all categories to produce the final score, will make for much more informative reviews, as each reader will be able to choose a sword that matches their own personal tastes, whilst knowing the TRUE overall picture as well. And who's to say ALL the categories should be equally weighted? In other words is "structural integrity" more important than "aesthetics"? And what should that weighting be? It's all still pretty subjective, except for Suvurov chopping pine or Paul whacking a 55 gal drum, both of which communicate truck loads.
|
|
Deleted
Deleted Member
Posts: 0
|
Post by Deleted on Oct 11, 2007 23:10:56 GMT
With the way I interpret Paul's Review Template and Guidelines, the items you mentioned are included in the suggested scoring process. What about aesthetics, or quality control? I think the inclusion of these two extra categories could vastly improve the way in which we help others understand our own impression on the swords we review. I think that fit and finish included these two items. How well the sword is assembled, or its fit, is a direct result of the quality control. The finish of the sword, or its aesthetics, are a measurement of how pretty it looks. Finally, our final score is decided by ourselves, but perhaps it would be better if we got the calculator out and actually worked out an average of the scored we give in each area, so as to give a truer representation of the sword. Paul states to do exactly what you suggest. Historical Accuracy: ?/5 Fit and Finish: ?/5 Handling: ?/5 Structural Integrity: ?/5 Value for Money: ?/5
OVERALL: ?/5 (Add Fit and Finish, Handling, Structural Integrity and Value for money scores together and divide by 4). Take the total of the scores for Fit & Finish, Handling, Structural Integrity, and Value then divide by 4 for the average rating.
|
|
Deleted
Deleted Member
Posts: 0
|
Post by Deleted on Oct 11, 2007 23:30:29 GMT
I think one way around the proportion issue might be to eliminate the overall score and keep categorical scores. I certainly won't buy a sword because it was a 4 overall, the meat of the review is what makes or breaks that sword for me.
Aesthetics and QC could be easily go under fit and finish. Maybe a more specific breakdown of criteria for each category? That is to say, what if there was a more rigid guideline for the factors that go into each rating, for instance structural integrity taking into account A: Temper B: construction C: tightness of fittings D: flexibility E: edge geometry. Something along those lines.
Just brainstorming, that may be alot more depth that people want to go into. ;D
|
|
|
Post by ShooterMike on Oct 12, 2007 2:31:32 GMT
Maz, I like those ideas. But I think it will be difficult to come to agreement with what all the testing protocols should be. Gus Trim and I have discussed and emailed back and forth extensively over this. But the bottom line is that everyone doesn't have the facility and materials to to a thorough test.
However, nothing would please me more than to have a set of solid guidelines that say:
A. Cut this in this way. B. Cut this other thing like this. C. Thrust into this thing like this. D. Look at these points an rate them like this. etc, etc...
Even if I don't agree with them all. At least having something iron-clad and objective might eliminate or at least curtail some of the subjectivity.
And don't get me wrong. I am as subjective as the next. I just find myself looking over old reviews I've done and thinking "Hmmm...Well Damn! I wouldn't rate it that way now!"
Anyone got more ideas?...
|
|
|
Post by rammstein on Oct 12, 2007 2:50:25 GMT
What I don't like (and I've said this before) is how someone who is completely unfamiliar with swords gives a rating of say, 4/5 to a sword thats a half a step up from an SLO, and thinks it's the greatest sword in the world simply because that person has never had anything else to guage their opinion by. I've seen swords here that have gotten outstanding ratings that I feel would be well below a 2/5 in my book....of course, I don't own most of these swords, so I'm going on glaringly obvious things, alone.
|
|
|
Post by Brian of DBK on Oct 12, 2007 3:01:20 GMT
There is no perfect science to the 'review'. It is all subjective, and mostly based on opinion. Much like the basis of a forum, we're all opinionated, and we like to respond as much as we like to read. The idea here is to provide users with more insight on not only a sword of particular interest, but insight on what others do with their swords. Maybe we can refine things a little, but lets not be anal about it either.
|
|
Deleted
Deleted Member
Posts: 0
|
Post by Deleted on Oct 12, 2007 4:15:32 GMT
My pine table tops thank you Larry I understand the desire for definitive guidelines, but I personally wouldn't want to see it stray much from the way it is now. I know a lot people put a lot of stock in how a sword cuts a plastic bottle, or a water jug... its just not for me and I wouldn't want to be required to do so just to comply with a ratings scale. I figure that since I show you what I am cutting (and describe it), all of you can make your own judgements if my assessment of structural integrity is valid. Likewise, with enough photos, on any of the other assessment areas (except possibly value for the money) For me, a great part of testing the swords the way I do is about finding their failure points-- not what they do well, but how they survive the imperfect user (i.e. me). If they bend on a bottle, then I don't want it. If improper form is going to cause the death of the blade; likewise. Why? Because I figure any sword used in the heat of battle will probably be used with nothing akin to "proper form." I view my swords as potentially battle ready replicas, so they need to act like it. Other people will say I'm crazy (and they may be true ), so I say let them test as they will. I'll judge their reviews based on my interpretation of what they show, not the numbers they plug in, because all is subjective. Just my opinion, for what its worth.
|
|
admin
Site Admin
Posts: 2,089
|
Post by admin on Oct 12, 2007 9:16:24 GMT
I agree that it is time we have another look at the ratings system... As some of you may recall, many moons ago, I was the only dude doing the reviews on SBG (though Kreigswert was a very close second!). So of course, it was easy to have a fairly comparative scale and weigh it all up accordingly. Naturally, this format of 'the Paul Southren show' was not intended to go on as a one man band forever. But the ratings system has only ever been overhauled once - and it was actually streamlined... Sometimes when I do a review myself I feel that some factors are missing. For example, some swords have great fittings - but a so so blade, or the other way around. So I have to kind of average that within the 'fit and finish' - as well as adding QC factors... Another thing that I wanted to add was 'sharpness'. But again, that is fairly objective unless you set some standard cutting mediums that DON'T require the human factor. (I am an 'ok' cutter. But if we gave a sword to, say, James Williams or Toshishoro Obata, I am sure he could make an iaito cut a double rolled Mugen Dachi Tatami mat). I suppose at the end of the day, my own thoughts are along the lines of - the body of the review, pics and videos will tell you more than any set of pure statictics will. The stats are really just a very basic summary of the reviwers take on that particular sword. Sure, it ain't perfect. And it could use some more refinement. I am really enjoying the ideas appearing in this thread (+1 to Matt993 for bringing it up). I really want SBG to cover the whole spectrum of enthusiats - which I think we are more or less doing (we have from teenagers to retirees) - and I think, well, that is pretty darn cool. But any more ideas on how to make a more reprentative scoring system, while keeping it simple and accessible to all are more than welcome!! ;D
|
|
|
Post by Matt993f.o.d on Oct 12, 2007 11:49:16 GMT
I must confess I had forgotton about the 'fit and finish' catgory. It does contain the two categories I would like to see, but my problem with it is that it bunches it too close together. For example, like Paul says, a sword could have a great finish but poor fittings, or great fitment and a cruddy finish. It becomes difficult to grade it properly for each aspect.
Something like Maz has suggested would be more along the lines of what I would like to see (a breakdown of the catgory).
As for reviews being subjective, that is my point. We each value different things in the swords we like, and the addition of extra categories will help us better select the features we like best when choosing a new sword. By lumping more than one feature into a category (fit and finish, for example) we are limiting the ability of those who, for example, like a well put together sword better than a well finished sword, to choose their favoured characteristic. See what I mean?
|
|
|
Post by YlliwCir on Oct 13, 2007 18:06:45 GMT
All good food for thought, gentlemen. For myself, I wouldn't like any hard and fast rules set for reviews. I like to follow the template as well as I can but I'm still a rookie at this. So I like having some leeway. It also depends on the sword I'm reviewing as to what I do to it. For excample I wouldn't cut up furniture with the Eddy3 the way I did with the VA Viking. I've seen some reviews on other forums that give a lot of technical information, most of which is over my head, tho I understand more in depth people like that. These reviews to me are missing the "everyman factor". I like pictures and movies and what they put the sword through. I got to admit I kind of skim over that "in conclusion" rating score when I read a review because I realise thats an opinon and I know mine gets biased and as has been pointed out changes with time. The bottom line for me is, even with it's inconsistencies, this is the most informative review board I've seen.
|
|
Deleted
Deleted Member
Posts: 0
|
Post by Deleted on Oct 13, 2007 20:36:07 GMT
I think we could give the swords a kind of summary rating at the end of the review. 0/5 doenst really give much room. there can be 2 swords that recives a rating of 4/5 while one of the swords may be much better than the other, just not high enough for 5/5.
I think we should extend the maximum amount of "score" a sword can achieve in the "overall" category (which will not be based on the other scoring categories) to say 0/100?
whith the swords i mentioned earlier might've got 60/100 and 79/100 in which we can see a large difference between the score of the swords.
and the 0/100 score will be determined by the users overall feeling of the sword.
if we take a katana for example. a nihonto would probably land above 95/100 while a cheap ebay wallhanger would get below 10/100
Sounds like a good idea?
|
|
|
Post by rammstein on Oct 13, 2007 20:41:42 GMT
I actually like skalman's idea...
|
|
|
Post by Brian of DBK on Oct 13, 2007 21:38:14 GMT
Let me highlight on that. I think swords should be rated according to price bracket as well. We have sub $100 swords, $100 to $200 swords, $200 - $300 and up. I think the ratings should flex according to price bracket. A 4/5 sub $100 sword doesn't compete against a 4/5$300 sword, so the comparison rating isn't relative. But in the same price bracket, the ratings are much more definitive and comparable. Agreed?
|
|
Deleted
Deleted Member
Posts: 0
|
Post by Deleted on Oct 13, 2007 21:47:45 GMT
Yeah, it's a much more accurate system. Instead of 4.5, something might get 85 or 90%
Matrix's Idead has merit too. A $79 musashi might be a great sword... in it's price range.
A Cheness Kaze by comparison might have the same score, but be light years ahead.
|
|
Deleted
Deleted Member
Posts: 0
|
Post by Deleted on Oct 19, 2007 23:11:06 GMT
As a reader, the "Historical Accuracy" rating seems to be a bit of a throwaway element. In the reviews I have read, which does not remotely approach all of them, ratings of 4 are handed out without much of a reason being given. I know this is not myArmoury, but the thing I appreciate about their reviews is that they provide a background on the weapon type being reviewed. This helps form a basis for evaluating the historical accuracy of the item in the review.
Also, what about the sword is historically accurate? The general appearance? That it is sharp? Is it made in a way that approaches how a period sword would have been assembled?
Is the sword's accuracy rating based on how accurate it is for a sub-$300 price point, or production swords in general?
I apologize if these questions have already been answered at some point.
|
|
|
Post by rammstein on Oct 19, 2007 23:12:37 GMT
Thanks Jonathan....Karma well deserved.
Personally I find historical accuracy, both in looks and function, to be of PARAMOUNT importance when reviewing a sword...
|
|
admin
Site Admin
Posts: 2,089
|
Post by admin on Oct 24, 2007 11:02:57 GMT
A good point raised there Jonathan.
It is true that the focus of SBG reviews is quite different than at My Armoury. But it would not hurt to at least provide more info on how a sword compares to a historical version if that is what it is based on.
Are there any specific reviews that you feel the historical accuracy has been given an overly high rating without explanation? I would be happy to revisit them and perhaps add something more into the introductory blurb ala My Armoury.
|
|