RCP 2021-002

TED COCHRAN said : Report 6 months ago

I am opposed to this, for several reasons.

A well-thought out scoring rubric constrains contestants to perform to its requirements, assuming they want to do well. If the requirements are narrow and well-thought out, the rubric may be helpful. Given that the stated purpose of NAR R&D competition is to stimulate new concepts, approaches, and ideas in:

• Advancing the state-of-the-art of hobby rocketry; or
• Using hobby rocketry as a research tool; or
• Preserving the history of hobby rocketry,

the event may be too broad to impose a rubric like this. The proposed rubric appears to favor engineering projects and experiments over everything else, leaving no room for projects that advance the state of the art of hobby rocketry in other ways, let alone historical preservation.

Even if one were to accept that R&D needs a rubric, one would hope that it would be developed in a principled way. Why, for example, does a problem statement count as much as an engineering solution? Why, for that matter, do the weightings break down the same way for experiments as they do for engineering projects? Shouldn't analysis count more for a study, and the solution count more for an engineering project? [And if there is disagreement, that demonstrates the point I'm trying to make: We can't have a rubric without a well-defined set of requirements in a constrained problem domain. If we're arguing over the weights, let alone the dimensions themselves, we aren't yet ready to impose a rubric.]

Perhaps an R&D study is in order to evaluate alternatives for rewarding thoughtful contributions to the the state of the art of hobby rocketry!

[As background, I have spent decades developing rubrics and scoring methods and using them on everything from Lego Robotics to student rocketry experiments to TARC Presentations to Baldridge Award evaluations to a variety of interesting professional assessment activities. Not one of these used with four equally-weighted dimensions plus points for presentations.]

Jonathan Rains said : subscriber Report 6 months ago

 

I concur that this is a big lift for competitors in A Division.  It's one thing to dumb down and event, another to set a bar that is unobtainable by many of this age. The best entries are those managed by the entrant, not the parent.

 

MATT STEELE said : subscriber Report 6 months ago

Don - In my opinion, the "historical research" class of reports does not belong in this competition. Hence, no accommodation for them.

Donald Carson said : microsite admin Report 7 months ago

One other considerations is that this proposal would require an oral presentation for every R&D entry, even those token entries that were submitted to get some "flight points" (we all know that happens). Currently, only the top contenders (a half dozen or so) are invited to present orally in each division. That tends to fill two evenings as it is. Oral presentations for every entry will present a scheduling issue at a NARAM. Pity the judges. 

Robert Zurek said : subscriber Report 7 months ago

As a past state and regional science fair judge, I agree a rubric helps foster consistency between judging panels and from one year to the next. Science fairs have as their goal to teach students scientific processes. NAR R&D was not put in place to teach the member how to set up a science fair project buy rather to advance the state of the art of model rocketry. (art in the sense of technology and innovation) By the definition of NAR R&D in the sporting code,  "The purpose of this event is to stimulate new concepts, approaches, and ideas in: Advancing the state-of-the-art of hobby rocketry..."  At the very least, the rubric should represent those desires in its judging. IMO there should be categories in the rubric related to the originality of the experiment, the usefulness of the results to the membership, and the relevance of the research to model rocketry. This rubric seems to focus more on the process of conducting a research project, than on the value of that research. While I believe the concept of a rubric is a good idea, I do not think I can support the limited scope of this particular rubric for this competition as it is defined. 

PATRICK PETERSON said : subscriber Report 7 months ago

A similar proposal was co side red a few years ago.  Although there might be a way to do this that can be flexible enough to handle all the different kinds of R&D, I see two major problems with the current proposal:  First, the point allocation doesn't seem that well thought out and doesn't seem flexible enough to handle the potential variety of different kinds of R&D.   Second, the most important reason for even having R&D as a competition event is to encourage real science and innovation, and so judges have to have a way of rewarding that.  In any given year, you might have a project that is weak on some of the conventional categories the proposal identifies, yet provides such stunning and remarkable insights and innovation that it deserves first place regardless of certain flaws.  A scoring guide is not in principle a bad idea, but it needs to be more flexible than this and be able to reward the right things.  I just think it needs more work.  -- Patrick Peterson

CHAD RING said : editor Report 7 months ago

 

Unfortunately, there is a need for this 'guide structure' in scoring this event.  Prior to NARCON 2017, there would have been no need for Matt to write this RCP.  But it is a valuable addition to the rules for this event, even when it finally sees the day it no longer has a rule section number in front of it.  That day can't come soon enough....  :)

 

BERNARD CAWLEY said : subscriber Report 7 months ago

As someone who has only done one R&D project for a NARAM, the idea of having some kind of structured scoring system and especially some kind of feedback to the contestant required appeals to me. I have no idea how I scored (other than my placement) and no idea how the scores were arrived at for that one outing.

Also — I want to echo Don Carson’s comment about provisions for historical research in the scoring rubric, if only because I’m contemplating such a project in the future.

Donald Carson said : microsite admin Report 8 months ago

I like the structured guidelines and the feedback for the contestants. 

I have a couple of concerns. First, the rubric does not address the class of R&D reports that represent historical research. Those have been established as viable R&D projects and should have a rubric as well. Secondly, I think the rubric doesn't fairly address the A and B Division R&D reports. The requirements for younger ages should be less stringent than that for C Div. and Teams, otherwise you will end up with unduly low scores for projects that are really reflective of the abilities of younger contestants. The result may well be either a more limited spread of scores in A&B divisions or adults "helping" the younger contestants to make sure that "all the boxes are checked."

DOUG HILLSON said : subscriber Report 8 months ago

I would recommend adding a requirement that the R&D Judging Rubric have space for optional written feedback from the judges and be in an electronically editable format. This update should be made to the exact wording for rule revision and reflected in the proposed rubric.

Providing the judges an opportunity to supply written feedback, in addition to any oral feedback during presentations, could improve the quality of R&D projects. The scoring system without written feedback still has the potential to be vague without improving future projects or reducing miscommunications.

I personally don’t have a strong opinion on the proposed scoring system. It is my understanding from judging state science fair projects, that the scoring system is intended to normalize scores across a large number of judges who may not judge every entry. Obviously that’s not the scenario we have in NAR R&D competition.