Select one or more videos and choose Export from the File menu to export the videos to a new format outside your library. New videos will be created in the folder shown in the "Export to folder" field at the top of the dialog.
Click the "Create web site" checkbox to also export HTML files. A modern browser with H.264 video support is required.
For more about HTML5 video see: http://diveintohtml5.org/video.html
See also: export formats.
Clipstart comes with a plain default HTML template, installed in /Users/your-user/Library/Application Support/Clipstart/Themes. If you duplicate the "Basic" folder and customize it, your new theme will show up as an export choice in Clipstart.
Themes support special "{{" tags to dynamically include video names, tags, and other information, as well as looping and conditionals as you might expect in a very simple scripting language. See the basic theme for example usage and available video properties.
If you make an interesting theme I encourage you to share it on the Riverfold forums, and I'll highlight it on the web site.