Skip to Sidebar Skip to Content

National Association of Rocketry

  • Home
  • Join NAR
  • Site Map
  • Contact Us
  • My Membership
  • About NAR
    • Overview
    • Organization Awards
    • Organization Contacts
  • Donate

Research & Development

« Miscellaneous Events
  • NAR Rocketry Competition
  • Altitude Events
  • Craftsmanship Events
  • Duration Events
  • Miscellaneous Events
  • Techniques and Tips
  • Research & Development
  • Spot Landing

  • by Joyce Ann Guzik, NAR 37191

    Editor’s note: Joyce has judged the R&D event at fifteen NARAMs. She is a staff scientist at Los Alamos National Laboratory and holds a Ph.D. degree in Astrophysics.

    I thought it would be good to give you an idea of what the judges are looking for and, conversely, what the judges have often found lacking about R&D reports, and also give you some useful tips and advice.

    First, it is important to read the R&D rules in the Model Rocket Sporting Code on this event. You must submit three (3) copies of your R&D report for judging, and starting in 2016 you are required to submit the report in electronic format two weeks before NARAM in order to give the judges time to review it before NARAM. The rules give a list of elements that must be included in a report in order to qualify for the event, such as stating how much it cost to do your project. You must also submit a separate 250-300 word written summary of the report. You must be prepared to give an oral presentation before the judges if your project is in contention for 1st through 4th place.

    PROJECT IDEAS
    According to the Sporting Code, an R&D project is supposed to either advance the state of the art of model rocketry, or use model rocketry as a research tool. Follow your curiosity. A good project has an idea that shows creativity and originality, is feasible to execute in a reasonable amount of time, and (sometimes fortuitously) leads to a result that is useful, or advances everyone’s understanding.

    Is there something that has been mysterious, that you have always wanted to understand? Have you wanted to improve the performance of your rocket? Is there something that has always annoyed you about your rocket (misfires, the way your parachute tangles or won’t deploy, red barons, not being able to find it after launch, the difficulties of altitude tracking, lack of guidance, safety concerns, etc.) for which you might have a novel solution? Is there something that you could use model rocketry for (e.g., taking atmospheric science data, taking aerial photos for a particular use, developing a science curriculum to teach some aspect of physics or math or chemistry, etc.)? For examples of what R&D projects have been done in the past, you can search the bibliography that is posted at the bottom of this page. (Alternatively, click here for a list of R&D Reports available to NAR members).

    START EARLY!
    Don’t plan on doing your project a few days before NARAM! The winning R&D projects in the upper divisions often take months to over a year to complete, with multiple aspects/parts/approaches. Building your test rockets or equipments, testing, allowing time for failures and revising your approach, and getting a statistically significant amount of quantitative data from experiments and flight tests (see below) all take time.

    Keep a logbook chronicling your progress, ideas, data, and observations. It will be useful when it comes time to write your report.

    THE WRITTEN REPORT
    Be sure that everything listed in Pink Book rule 63.5 is included in your report. Your entry could be disqualified if it does not contain all of the required elements in some form. Include an introduction about why you chose this project and what was your initial (and sometimes evolving) objective for the project.

    A written summary of your report is also required. This is not simply a repeat of the introduction, or an advertisement for the rest of the project-it is a succinct summary of your objective, a brief statement of the means you used to meet it (e.g., calculations, flight tests, engineering tests, or other data gathering), and a statement of the conclusions and significant results (What did you learn? Did you meet the objective?). If a reader had time to read only your summary, what would you want him/her to know about your project?

    Keep a bibliography and list the sources that you used for reference. Do go to the library (surprisingly, there is stuff there that isn’t on the web!), look at the web, and ask other experienced modelers. But make sure your writing is original-DO NOT cut and paste entire paragraphs from reference sources on the web or from books unless you have a purpose for them in the context of the report. Material taken from a source should appear within quotes in your report, and a reference to the original source should appear in your bibliography. Even if you paraphrase material, or put it in your own words, you need to reference the original source. Also, stick to the point of your project, and do not include extensive irrelevant information or descriptions. You will not earn extra credit for padding out your project with more pages.

    Periodically during your preparations, you should try to step back and think carefully about your experimental setup and think about what else could account for or affect your results (was Rocket B heavier than rocket A by a little? Did Rocket B have slightly crooked fins and so not fly as straight or have more drag?). Include these thoughts in your report. Even if you don’t know how to correct for such systematic errors, just doing some thoughtful analysis and itemizing other possible causes for the results will earn you the esteem of the judges!

    It would also be good after the conclusions in your report to include a retrospective on what you would do differently if you could start over, your thoughts on follow-up work that could be done to improve the project, or other ideas that this project inspired.

    Please have several other people read your report before you enter it in the R&D event. Have them make suggestions for improvements. Find out what things they thought were confusing or missing. Also, be careful to eliminate all of the grammar and spelling mistakes (a good proofreader helps here too!).

    RESULTS
    One thing that the judges are especially looking for is that the contestant is not drawing incorrect or unsupported conclusions from their observations and experimental results.

    This can arise for two reasons. First, the contestant may not have enough controls or checks in his/her experimental process. For example, if you are doing a lot of flight tests with several different models, you need to check for and record possible differences between the models. Do they have the same mass, surface finish, fin area, frontal area, etc.? Also, follow procedures that help to ‘randomize’ the effect of influences that you may not consider or can’t control. For example, if you are comparing the performance of A vs. B engines using what you think are two identical models, don’t fly all of the A engines in one model, and all of the B engines in the other model; divide the two types of engines evenly between the models. Similarly, if you are comparing two different types of models, don’t make all the flights of model #1 in the cool, clam morning, and then make all the flights of model #2 in the hot, windy afternoon; mix the flights together so that external changes affect both groups equally. The second reason people draw unsupported conclusions is that they don’t consider whether any differences in their results are statistically significant or not. For any quantitative data it is a must to have some statistical analysis. Most importantly, you need to have enough trials of an experiment to get statistically significant results. One to three trials is much too few to get significant results; ten to twenty would be much better. At a minimum, you should calculate the ‘mean’ (average) and the ‘standard deviation of the mean’ or all sets of experimental data you want to compare.

    The short technical report Basic Statistics for R&D by NASA astronaut and rocketeer Dr. Jay Apt, available from NARTS (www.nar.org/narts), discusses in detail the calculation of mean and standard deviation of the mean. This is an estimate of how far your calculated mean is likely to differ from the true mean that would be found by doing a very large number of trials. There is a 68% chance that the true mean lies in the range of one standard deviation on either side of your sample mean; the chance is 95% that it’s in the range of two standard deviations.

    If the standard deviation error ranges of two sets of experimental data overlap, then you cannot claim there is a statistically significant difference between the two sets. For example, as Dr. Apt points out, if you have a set of trials of rocket A with an average altitude 320 ± 20 meters, and a set of rocket B with 310 ± 20 meters, you cannot conclude that rocket A reaches higher altitudes than rocket B! Including more trials in a data set gives a narrower error range for the standard deviation and can result in you being able to determine a statistically significant difference that fewer trials will not reveal.

     

    Figure 1. Rocket data example spreadsheet

    Rather than duplicate Dr. Apt’s report by demonstrating how to calculate the mean and  for your data, we show you how to do it in an Excel spreadsheet (see Figure 1). The bottom of the figure shows what formulas are in the cells of column B (the same formulas are copied across into column C). The “track lost” entry in cell C6 is there to show that the formulas ignore cells that are blank or contain non-numerical data.

    The spreadsheet shows two sets of rocket altitude measurements and the mean (average) of each set. But is there a significant difference between the means? Using the  values from row 10, we can see that the error ranges 191.0 ± 5.2 and 202.0 ± 4.7 do not overlap, so the data are significantly different (at least to ‘one sigma’-one standard deviation of the mean). The difference in the results would be more convincing if the data were different at the ‘two sigma’ level (95% confidence) but they are not, since doubling the error ranges causes them to overlap. By the way: if the we only had the first three flights in each set, the results are 191.7 ± 11.1 and 204.3 ± 8.0, which does not show a clear difference between the two sets since those ranges overlap.

    The Student’s t-test formula in cell B13 is another statistical test that Excel can perform. The result shown tells us that there is only a 13% chance that the two data sets come from the same population, another indication that the data are significantly different.

    ORAL PRESENTATION
    In addition to the written report, the contestants that are in line to place must do an oral presentation in front of three NARAM R&D judges and a friendly audience of the interested NARAM attendees, and answer questions following their talk from the judges and audience. The presentation can significantly affect the judges’ opinions, but remember that you need to do well enough to impress the judges on the written report to earn the chance to present the oral in the first place!

    Prepare your oral talk well: Have a Powerpoint digital presentation summarizing your report (proofread it!), and use large font sizes for the text so people in the back of the room can see them.  Check with the event organizers to ensure that you know whether you are supposed to bring a laptop of your own or just a digital file to load onto their laptop.  Have some props as well (samples of your rocket, apparatus, etc.) to show the judges and pass around. Practice in front of your family and friends two or three times before coming to NARAM and ask them for their honest constructive criticism regardless of how much it hurts your ego!

    The R&D rules state that the oral presentation shall not exceed 15 minutes in length, but the constraints of time at NARAM usually lead to the judges to request talks with a length of 5 or 10 minutes, so you should be prepared for either length. Contestants that go on past the time limit may be cut off from necessity without finishing. One viewgraph per minute is a good rule of thumb. You could have an additional 5 minutes of material in reserve if it turns out there is time for a longer format. The judges and audience will not interrupt you with questions during your presentation, but there will typically be five minutes of questioning after you have finished your talk. Practice answering questions from your family and friends.

    The judges may ask you to demonstrate some aspect of your project on the field on Friday, so be prepared for this if you think your project might warrant it.

    Good luck! We are hoping to see many outstanding projects at NARAM.

    Related Documents:

    Basic Statistics for R&D (Apt) PDF May 28, 2014, 9:31 pm 190 KB
    Rocketry Technical Bibliography (Apr 2015) PDF April 12, 2015, 10:53 pm 215 KB
    • Return to home
    • Join the NAR
      • Home
      • Join NAR
      • Site Map
      • Contact Us
      • My Membership
      • About NAR
      • The American Rocketry Challenge
      • Model Rocket Info
        • The Sport Rocketry Hobby
        • Model Rocket Safety Code
        • Understanding FAA Regulations
      • High Power Rocketry
        • High Power Rocketry Certifications
        • High Power Rocket Safety Code
        • Understanding FAA Regulations
        • Filing for FAA Launch Authorization
        • High Power Resources
          • High Power Technical Reports
          • NAR HPR Certified Member Search
        • The NAR Rocket Science Achievement Award
      • Educational Resources
        • Online Mentoring Resources
        • NARTREK Skills Program
        • The NARTREK Cadet Program
        • 4-H
        • Air Force JROTC
        • Civil Air Patrol
        • Rocket Teacher Certification Program
        • Grants, Scholarships, and Recipients
        • Educator’s Newsletter Archive
      • Find a Launch
        • General Launch Information
        • Launch Calendar
        • Find a Contest
        • Register an NAR Launch
      • Local Clubs
        • NAR Club Locator
        • Find an NAR Club Launch
        • Submit a Local Club / Submit Changes
        • Section FAQ’s
        • Section Guidebook
      • Insurance Questions/Claims
      • Safety
        • Safety Codes
        • Model Rocket Safety Code
        • High Power Rocket Safety Code
        • Radio Control Rocket Glider Safety Code
      • Rocket Motor Information
        • Certified Motors
          • Motor Related Lists
        • NAR Motor Testing
          • NAR Motor Testing Notices
        • Manufacturer Notifications
        • Report a Malfunctioning Motor
      • Contest Flying – NAR Competition
        • National Scoreboard
        • US Team Selection Scoreboard
        • Find a Contest
        • Register an NAR Launch
        • Competition Results Entry
        • Team Registration
        • Teams List
        • Contest Events
        • Competition Meets
        • US National Model Rocket Performance Records – 8/2/19
        • International Competition
        • Competition Guide
        • U.S. Model Rocket Sporting Code
        • Rules Revision Process
      • Sport Rocketry Magazine
        • Magazine Submissions
        • Write It and Get Paid
        • Advertise in Sport Rocketry
        • Sample Issue
        • Contact Sport Rocketry
      • Member Resources
        • Members Only Forum
        • Magazine Archives
          • Sport Rocketry Magazine
          • eRocketeer Digital Editions
          • Member Guidebook Archive
          • American Spacemodeling Magazine
          • The Model Rocketeer Magazine
          • Model Rocketry Magazine
        • NARTREK Skills Program
        • R&D Reports
        • Technical Bibliography
        • NAR Insurance
          • Insurance Questions
          • Proof of Insurance
          • NAR Insurance Claim Form
        • NAR Association Documents
        • Promotional Materials
      • NAR Products
      • Website Fixes
      • Donate
      • What’s My NAR Number?
      • Log out

    • High Power Rocket Certification

      Design and Fly High Power Rocket Vehicles Learn More

    • Local Rocketry Clubs

      Connect with Proven Rocketry Experts Find A Club Today

    • Members-Only Forum

      Engage in meaningful conversations about rocketry topics that matter. Get Connected!

    • Sport Rocketry Magazine

      Six 48 Page Issues a year with NAR Membership!


    • Contact Nar

      National Association of Rocketry
      P.O. Box 407
      Marion, IA 52302

      Call: 319-373-8910

    • Join NAR

      The best way to get more out of the exciting hobby of rocketry is to connect with other flyers. The National Association of Rocketry can get you connected to the cutting edge of the hobby so you can learn how to fly higher, faster, and better. As an NAR member you will get:

      • Access to launch sites through local NAR clubs
      • The bimonthly color magazine "Sport Rocketry"
      • Insurance coverage for your flights
      • Certification for high power rocket flying

      Join Nar

      © Copyright YEAR National Association of Rocketry | Privacy Notice

      • Home
      • Join NAR
      • Site Map
      • Contact Us
      • My Membership
      • About NAR
      • The American Rocketry Challenge
      • Model Rocket Info
        ▼
        • The Sport Rocketry Hobby
        • Model Rocket Safety Code
        • Understanding FAA Regulations
      • High Power Rocketry
        ▼
        • High Power Rocketry Certifications
        • High Power Rocket Safety Code
        • Understanding FAA Regulations
        • Filing for FAA Launch Authorization
        • High Power Resources
          ▼
          • High Power Technical Reports
          • NAR HPR Certified Member Search
        • The NAR Rocket Science Achievement Award
      • Educational Resources
        ▼
        • Online Mentoring Resources
        • NARTREK Skills Program
        • The NARTREK Cadet Program
        • 4-H
        • Air Force JROTC
        • Civil Air Patrol
        • Rocket Teacher Certification Program
        • Grants, Scholarships, and Recipients
        • Educator’s Newsletter Archive
      • Find a Launch
        ▼
        • General Launch Information
        • Launch Calendar
        • Find a Contest
        • Register an NAR Launch
      • Local Clubs
        ▼
        • NAR Club Locator
        • Find an NAR Club Launch
        • Submit a Local Club / Submit Changes
        • Section FAQ’s
        • Section Guidebook
      • Insurance Questions/Claims
      • Safety
        ▼
        • Safety Codes
        • Model Rocket Safety Code
        • High Power Rocket Safety Code
        • Radio Control Rocket Glider Safety Code
      • Rocket Motor Information
        ▼
        • Certified Motors
          ▼
          • Motor Related Lists
        • NAR Motor Testing
          ▼
          • NAR Motor Testing Notices
        • Manufacturer Notifications
        • Report a Malfunctioning Motor
      • Contest Flying – NAR Competition
        ▼
        • National Scoreboard
        • US Team Selection Scoreboard
        • Find a Contest
        • Register an NAR Launch
        • Competition Results Entry
        • Team Registration
        • Teams List
        • Contest Events
        • Competition Meets
        • US National Model Rocket Performance Records – 8/2/19
        • International Competition
        • Competition Guide
        • U.S. Model Rocket Sporting Code
        • Rules Revision Process
      • Sport Rocketry Magazine
        ▼
        • Magazine Submissions
        • Write It and Get Paid
        • Advertise in Sport Rocketry
        • Sample Issue
        • Contact Sport Rocketry
      • Member Resources
        ▼
        • Members Only Forum
        • Magazine Archives
          ▼
          • Sport Rocketry Magazine
          • eRocketeer Digital Editions
          • Member Guidebook Archive
          • American Spacemodeling Magazine
          • The Model Rocketeer Magazine
          • Model Rocketry Magazine
        • NARTREK Skills Program
        • R&D Reports
        • Technical Bibliography
        • NAR Insurance
          ▼
          • Insurance Questions
          • Proof of Insurance
          • NAR Insurance Claim Form
        • NAR Association Documents
        • Promotional Materials
      • NAR Products
      • Website Fixes
      • Donate
      • What’s My NAR Number?
      • Log out