resolve_output_format function

Resolve the output format for an R Markdown document