---
title: "Creating prompt wraps"
output: rmarkdown::html_vignette
vignette: >
  %\VignetteIndexEntry{Creating prompt wraps}
  %\VignetteEngine{knitr::rmarkdown}
  %\VignetteEncoding{UTF-8}
---


### Creating prompt wraps

Using `prompt_wrap()`, you can create your own prompt wraps.
An input for `prompt_wrap()` wrap may be string or a tidyprompt object. If you
pass a string, it will be automatically turned into a tidyprompt object.

Under the hood, a tidyprompt object is just a list with a base prompt (a string) and a series of prompt
wraps. `prompt_wrap()` adds a new prompt wrap to the list of prompt wraps. Each prompt wrap is a list
with a modification function, an extraction function, and/or a validation function (at least
one of these functions must be present). The modification function alters the prompt text,
the extraction function applies a transformation to the LLM's response, and the validation function
checks if the (transformed) LLM's response is valid.

Both extraction and validation functions can return feedback to the LLM, using
`llm_feedback()`. When an extraction or validation function returns this, a
message is sent back to the LLM, and the LLM can retry answering the prompt according
to the feedback. Feedback messages may be a reiteration of instruction or a specific error
message which occured during extraction or validation. When all extractions and validations
have been applied without resulting in feedback, the LLM's response (after transformations
by the extraction functions) will be returned. (`send_prompt()` is responsible
for executing this process.)

Below is a simple example of a prompt wrap, which just adds some text to the base prompt:

```{r, eval=FALSE}
prompt <- "Hi there!" |>
  prompt_wrap(
    modify_fn = function(base_prompt) {
      paste(base_prompt, "How are you?", sep = "\n\n")
    }
  )
```

Shorter notation of the above would be:

```{r, eval=FALSE}
prompt <- "Hi there!" |>
  prompt_wrap(\(x) paste(x, "How are you?", sep = "\n\n"))
```

Often times, it may be preferred to make a function which takes a prompt and
returns a wrapped prompt:

```{r, eval=FALSE}
my_prompt_wrap <- function(prompt) {
  modify_fn <- function(base_prompt) {
    paste(base_prompt, "How are you?", sep = "\n\n")
  }

  prompt_wrap(prompt, modify_fn)
}
prompt <- "Hi there!" |>
  my_prompt_wrap()
```

Take look at the source code of `answer_as_boolean()`, which also uses extraction:

```{r, eval=FALSE}
answer_as_boolean <- function(
    prompt,
    true_definition = NULL,
    false_definition = NULL,
    add_instruction_to_prompt = TRUE
) {
  instruction <- "You must answer with only TRUE or FALSE (use no other characters)."
  if (!is.null(true_definition))
    instruction <- paste(instruction, glue::glue("TRUE means: {true_definition}."))
  if (!is.null(false_definition))
    instruction <- paste(instruction, glue::glue("FALSE means: {false_definition}."))

  modify_fn <- function(original_prompt_text) {
    if (!add_instruction_to_prompt) {
      return(original_prompt_text)
    }

    glue::glue("{original_prompt_text}\n\n{instruction}")
  }

  extraction_fn <- function(x) {
    normalized <- tolower(trimws(x))
    if (normalized %in% c("true", "false")) {
      return(as.logical(normalized))
    }
    return(llm_feedback(instruction))
  }

  prompt_wrap(prompt, modify_fn, extraction_fn)
}
```

Take a look at the source code of, for instance, `answer_as_integer()`, `answer_by_chain_of_thought()`,
and `answer_using_tools()` for more advanced examples of prompt wraps.

#### Breaking out of the evaluation loop

In some cases, you may want to exit the extraction or validation process early. For instance,
your LLM may indicate that it is unable to answer the prompt. In such cases, you can have your
extraction or validation function return `llm_break()`. This will cause the evaluation loop
to break, forwarding to the return statement of `send_prompt()`. See `quit_if()`
for an example of this.

#### Extraction versus validation functions

Both extraction and validation functions can return `llm_break()` or `llm_feedback()`.
The difference between extraction and validation functions is only that an extraction may
transform the LLM response and pass it on to the next extraction and/or validation functions,
while a validation function only checks if the LLM response passes a logical test (without
altering the response). Thus, if you wish, you can perform validations in an
extraction function.

#### Prompt wrap types and order of application

When constructing the prompt text and when evaluating a prompt, prompt wraps are applied
prompt wrap after prompt wrap (e.g., first the extraction and validation functions of
one wrap, then of the other).

The order in which prompt wraps are applied is important. Currently, four types of prompt wraps
are distinguished: 'unspecified', 'break', 'mode', and 'tool'.

When constructing the prompt text,
prompt wraps are applied in the order of these types. Prompt wraps will be automatically reordered if
necesarry (keeping intact the order of prompt wraps of the same type).

When evaluating the prompt,
prompt wraps are applied in the reverse order of types (i.e., first 'tool', then 'mode', then 'break',
and finally 'unspecified'). This is because 'tool' prompt wraps may return a value to be used in
the final answer, 'mode' prompt wraps alter how a LLM forms a final answer,
'break' prompt wraps quit evaluation early based on a specific final answer, and 'unspecified'
prompt wraps are the most general type of prompt wraps which force a final answer
to be in a specific format.

#### Configuring a LLM provider with a prompt wrap

Advanced prompt wraps may wish to configure certain settings of a LLM provider.
For instance, `answer_as_json()` configures certain parameters of the LLM provider,
based on which LLM provider is being used (Ollama and OpenAI have different API parameters
available for JSON output). This is done by defining a `parameter_fn` within the prompt wrap;
`parameter_fn` is a function which takes the LLM provider as input and returns a list of parameters,
which will be set as the parameters of the LLM provider before sending the prompt. See
`prompt_wrap()` documentation and `answer_as_json()` for an example.

#### Configuring a prompt wrap based on the LLM provider or HTTP responses

`modify_fn`, `extraction_fn`, and `validation_fn` may take the
LLM provider as the second argument and the `http_list` (a list of all
HTTP responses made during `send_prompt()`) as the third argument. This allows for
advanced configuration of the prompt and the extraction and validation logic
based on the LLM provider and any data available the HTTP responses.

#### Configuring a prompt wrap based on other prompt wraps

`modify_fn`, `extraction_fn`, and `validation_fn` all have access to the `self`
object, which represents the `tidyprompt-class` object that they are a part of. This
allows for advanced configuration of the prompt and the extraction and validation logic
based on other prompt wraps. Inside these functions, simply use `self$` to access
the `tidyprompt-class` object. 

#### Handler functions

Prompt wraps may also include 'handler functions'. These are functions which are called
upon every received chat completion. This allows for advanced
processing of the LLM's responses, such as logging, tracking tokens, or other
custom processing. See the `prompt_wrap()` documentation for more information;
see the source code of `answer_using_tools()` for an example of a handler function.