1. Introduction
Technical demonstrations (demos) are experiments that show off a capability or phenomenon to either a live audience or through a video format. They differ from other experiments in that they are well polished so that a researcher knows roughly what will happen, and are designed to be viewed and understood by others. In this CommKit, we will go over the basics of how to design and execute a technical demo to maximize its impact.
2. Criteria for Success
- The audience leaves with a clear sense of the problem you’re trying to solve and how your solution performs
- Your demo engages the audience and allows them to get a more comprehensive understanding of how your technology physically behaves
- Your demo connects the audience to the context of your research and how it is useful for problems they understand
- Your audience walks away from your demo with a sense of increased interest and/or excitement about your work
- Your demo is conducted safely so that there is no risk of harming the audience or those involved with the demo
3. Audience and Purpose
Since technical demos convey information directly through sights and sounds, they can be more easily tailored to a wide variety of potential audiences when compared with other forms of technical communication. For example, written or oral presentations about your work can be effective for those who share your technical background, but might be less effective at invigorating a group of high schoolers. In contrast, the roar of an engine or the sight of a group of drones flying around can more easily connect with people of all backgrounds. The ability to connect with a wide range of audiences also means technical demonstrations can be used for many different scenarios.
The context of a technical demo includes the reason you are doing the demo and the format in which it will be shown. Demos may be done in a wide variety of contexts, including:
- to complete a checkpoint required by a funding source
- to attract potential funding sources for future research
- to generate interest in your lab/field for high school or undergraduate students or general public
and may be shown in either a live or video format.
Given the wide variety of possible reasons to run a technical demo, you may be presenting to audience members with a high degree of technical knowledge, none at all, or anywhere in between. Thus, you must carefully consider what information to present. Additionally, your audience may be large or small, possibly even single viewers of a video. These factors should be carefully considered and will help you craft a demo that can be effective for your audience. Some details on how to craft your demo for a specific audience are included in Section 6.
Regardless of your audience/purpose for doing a demo, there are two main goals that all demos should accomplish. Firstly, they should help your audience gain a better understanding of what problem you are working on and what capability/advancement your work provides. Technical demos give you an opportunity to connect with an audience in a more meaningful way than can be done with a typical paper or presentation. Seeing something in action allows your audience to see beyond equations and technical specifications to get a more intuitive understanding of how things work. Critically, this implies that your demonstration should work as intended. We will discuss this point more later, but for a demo to be effective, it must adhere closely enough to a plan to leave the audience with clear take-aways regarding the capabilities of your technology.
The second main goal of technical demos is to leave your audience excited about your technology and its possible implications/future directions. The goal is to show them something memorable that will stick with them after they’ve left the demo site. If you’re able to do this, your audience will be more likely to further engage with your work. This could take the form of reading/citing your relevant papers, considering you for funding, or being motivated to join your lab group.
4. Physical Setting
Another important thing to consider for your demo is the physical setting that it will be conducted in. Technical demos may be run in a wide variety of places, including in lab environments, fields/forests, on the water, or in the air. As shown in Fig. 1, outdoor vehicle demos in particular may range over large areas, making it a challenge to convey important information to your audience. In such cases, videos collected onboard the vehicle and/or graphical user interfaces (GUIs) that show its behavior can be especially helpful to show the audience what the vehicle was doing. The possibility of inclement weather introduces additional uncertainty for outdoor demos, so it is important to have a backup plan should the weather stop you from running your demo normally. It’s also important to consider your audience’s comfort by supplying things such as temporary shelters, water, and hand warmers depending on the conditions outside.
Outdoor demo location at the MIT sailing pavilion
Lab environments are more controlled than the outdoors, but come with their own set of challenges. Depending on the physical layout of your demo space, it may be difficult to ensure larger groups of people can see and/or hear the demo properly. An example of such a setup is shown in Fig. 2. In these cases it’s typically best to run the demo in small groups and have them rotate through, but this may not always be possible if there is an extensive process required to reset and rerun the demo. This is where video displays can again be useful to allow more people to experience the demo at the same time.
Indoor demo location in the MIT Kresa Center for Autonomous Systems
Depending on your audience/purpose, you may choose to do a video demo. While it can be harder to connect with your audience through this medium, video demos allow you to give your audience a uniform experience and eliminates the possibility of failure since you can just use a successful demo for the final video. Video demos also allow you to show a large number of experiments that might be difficult to show in a live demo due to the amount of time each one takes to set up. Despite the different format, video demos should seek to accomplish the same goals as a live demo. Thus, while PR journalism (e.g., MIT news) is a form of a video demo, there is a slight distinction as video demos are typically more targeted to a specific audience and give more technical insight about the experiments involved.
5. Risk Management
One context-dependent factor is the level of risk you want to take with your demo. Your most cutting-edge research may be more prone to failure. If your demo is intended to impress a funding source, it may be better to run a more conservative demo that has a higher likelihood of success – a failed demo in this situation can be harmful. In contrast, if you work closely with members of the audience or are recording your demo for a video, the opportunity for multiple tries means it may be appropriate to do harder things that are impressive if successful, but more likely to fail.
6. Best Practices
6.1. Design around your audience
One of the most important factors that determine the effectiveness of a demonstration is how well tailored it is to your specific audience. For more technical audiences, you may want to demonstrate your most advanced research (while maintaining consideration for the risk associated with such a demo, as described in Section 5), while simpler, lower-risk research may still connect with less technical audiences. Alternatively, you may still be able to show a less technical audience your most cutting edge research by altering the speech that accompanies the demo, which will be discussed in Section 6.3.
While it is good to provide some interactive component to your demo (i.e., wherein members of the audience can directly interact with the demo itself), in larger audiences or in a video format it may be infeasible to provide such an experience for all members of the audience.
The extent of the demonstrated experiments is also highly audience/purpose-dependent. If the purpose of your demo is to provide a progress update to someone you’ve been working with and is funding your project, they may want to see a variety of experiments demonstrating the nuances of what works and where the current technology is limited. This could involve a multitude of experiments that take some amount of time to set and reset. In contrast, for a group of visiting students, it is more important to provide a bite-sized experiment that leaves them with a clear bit of knowledge about your technology.
6.2. Preparation
Arguably the most challenging part of the demo is the preparation. Depending on how robust your experiment is, it may take a lot of effort to get it working consistently. For example, robotics experiments with complicated code may run into edge cases that require tweaks to their programming to ensure a smooth operation at the time of the demo. For especially challenging experiments, it may be acceptable to record the system’s behavior from a successful run, and simply rerun that recording on the hardware for the demo (just make sure you are transparent about doing this).
6.3. Execution
While the main focus of a demo is the physical phenomenon being presented, another important part is the speech that goes along with it. Before you run your demo, it’s best to explain some information about what you’re doing, why it is important, and what’s going to happen. This can also be a convenient time to encourage audience participation by introducing the experiment and asking the audience what they think will happen. Additionally, if a demo has a lot going on, it is good to pace things slowly enough to allow you to explain what’s going on while it’s running. Conversely, if it takes time to set up different runs within a demo, you can fill the time by having yourself or a co-researcher speak about the research and/or show videos while the setup is being completed. For video demos, annotations and voice overs can be added with video editing software such as Adobe Premiere Pro.
6.4. Safety
Regardless of what happens in your demo, the most important consideration for your demo should be the safety of yourself and your audience. Always make sure to wear the proper personal protective equipment and supply your audience with it as needed. Robotics experiments should always have a kill switch at the ready and flight experiments should either be conducted with a safety net in place or at a safe enough distance to ensure the safety of everyone involved. Engine demos should similarly have proper safety barriers in place. As necessary, be sure to work with MIT Environment, Health, and Safety to make sure your demo meets all MIT safety requirements.
7. Annotated Examples
7.1. Motion Planning Among Dynamic, Decision-Making Agents with Deep Reinforcement Learning
This video shows experiments with ground robots in environments with pedestrians. Voiceover accompanies the video to explain what is happening with each scene. First, at 0:00, the main idea of the demo (collision avoidance for a robot) is stated. We then get an introduction to the robot and its sensors at 0:02. Finally, the demo is shown at 0:17, where we see an aerial view of the robot in an environment with pedestrians. The voiceover explains what the robot is doing and the corner of the screen shows the robot’s understanding of its environment. This demo effectively portrays the capabilities of the authors’ algorithms without going into overly technical details of how they work. The use of perspective is also very effective – seeing the same demo from ground level would not be as effective at showing how the robot weaves in between people to reach its goal. One thing this demo could do better is to further motivate the problem, i.e., include information about why this type of behavior is useful.
7.2. Experimental Results for “Efficient Learning of Adaptive Policies via Tube-Guided Data Augmentation”
This video shows experiments done using drones and is meant to accompany a conference paper. This is clearly different than PR journalism [1] in that its entire focus is on the experimental results showing the drone’s performance in various scenarios. The lack of introduction is appropriate here since the video accompanies a paper that gives more context, but a standalone demo would need more introduction and context provided to help the viewers understand how the demonstrated technology improves on the state of the art. Regarding the video’s content, in the first experiment at 0:05, multiple camera views with on-screen annotations give a clear idea of what’s physically going on, without getting bogged down in the details of how the controller works. Again in the 2nd and 3rd experiments at 0:40 and 1:23, respectively, the video makes effective use of on-screen annotations to point out the novelty of their approach. If this demo were done in person, they would likely have to provide verbal explanations in place of the annotations.