Fine-tune LLMs with Weights & Biases
A Jupyter notebook that fine-tunes ChatGPT-3.5 using OpenAI's API with Weights & Biases integration to track experiments, models, and datasets.
Why it matters
Streamline the fine-tuning process for ChatGPT-3.5 and GPT-4 models by integrating with Weights & Biases for experiment tracking, model management, and dataset versioning.
Outcomes
What it gets done
Prepare and log datasets for fine-tuning to Weights & Biases Artifacts.
Initiate and monitor OpenAI fine-tuning jobs.
Synchronize fine-tuning job results and model details to Weights & Biases.
Track experiments, models, and datasets in a centralized W&B dashboard.
Install
Add it to your toolbox
Run in your project directory:
curl -fsSL https://spark.entire.vc/get/oai-gptfinetuningwithwandb | bash Steps
Steps in the chain
Start your Weights & Biases run. If you don't have an account you can sign up for one for free at www.wandb.ai
Set up your OpenAI API key for authentication.
Download a dataset from LegalBench, a project to curate tasks for evaluating legal reasoning, specifically the Contract NLI Explicit Identification task. This comprises a total of 117 examples, from which we will create our own train and test datasets.
Modify the base_prompt from the LegalBench task to make it a zero-shot prompt. Split it into training/validation dataset, training on 30 samples and testing on the remainder.
Save the data in train and test files first. Validate that training data is in the correct format using OpenAI fine-tuning documentation script. Log data to Weights & Biases Artifacts for storage and versioning.
Download training & validation files and save them to a folder called my_data. Retrieve the latest version of the artifact. Upload the training data to OpenAI for processing.
Define ChatGPT-3.5 fine-tuning hyper-parameters and start training. This takes around 5 minutes to train, and you get an email from OpenAI when finished.
Call openai wandb sync to log all un-synced fine-tuned jobs to W&B. Pass your OpenAI key as an environment variable, the id of the fine-tune job, and the W&B project where to log it.
Overview
Fine-tune ChatGPT-3.5 and GPT-4 with Weights & Biases
What it does
This notebook demonstrates fine-tuning ChatGPT-3.5 using OpenAI's API with integrated Weights & Biases (W&B) experiment tracking. The workflow covers dataset preparation, model training, and logging results. With a single command (`openai wandb sync`), fine-tuning jobs sync to your W&B dashboard. The notebook includes an example using a legal dataset from LegalBench's Contract NLI Explicit Identification task, comprising 117 examples split into training and test datasets.
How it connects
Use this notebook when you want to fine-tune ChatGPT-3.5 via OpenAI's API and track your experiments with W&B. Do NOT use this if you don't have an OpenAI API key and W&B account, or if you're working with models outside the OpenAI API ecosystem.
Source README
Fine-tune ChatGPT-3.5 and GPT-4 with Weights & Biases
Note: you will need an OpenAI API key to run this colab.
If you use OpenAI's API to fine-tune ChatGPT-3.5, you can now use the W&B integration to track experiments, models, and datasets in your central dashboard.
All it takes is one line: openai wandb sync
See the OpenAI section in the Weights & Biases documentation for full details of the integration
!pip install -Uq openai tiktoken datasets tenacity wandb
# Remove once this PR is merged: https://github.com/openai/openai-python/pull/590 and openai release is made
!pip uninstall -y openai -qq \
&& pip install git+https://github.com/morganmcg1/openai-python.git@update_wandb_logger -qqq
Optional: Fine-tune ChatGPT-3.5
It's always more fun to experiment with your own projects so if you have already used the openai API to fine-tune an OpenAI model, just skip this section.
Otherwise let's fine-tune ChatGPT-3.5 on a legal dataset!
Imports and initial set-up
import openai
import wandb
import os
import json
import random
import tiktoken
import numpy as np
import pandas as pd
from pathlib import Path
from tqdm.auto import tqdm
from collections import defaultdict
from tenacity import retry, stop_after_attempt, wait_fixed
Start your Weigths & Biases run. If you don't have an account you can sign up for one for free at www.wandb.ai
WANDB_PROJECT = "OpenAI-Fine-Tune"
Set up your API key
# # Enter credentials
openai_key = "YOUR_API_KEY"
openai.api_key = openai_key
Dataset Preparation
We download a dataset from LegalBench, a project to curate tasks for evaluating legal reasoning, specifically the Contract NLI Explicit Identification task.
This comprises of a total of 117 examples, from which we will create our own train and test datasets
from datasets import load_dataset
# Download the data, merge into a single dataset and shuffle
dataset = load_dataset("nguha/legalbench", "contract_nli_explicit_identification")
data = []
for d in dataset["train"]:
data.append(d)
for d in dataset["test"]:
data.append(d)
random.shuffle(data)
for idx, d in enumerate(data):
d["new_index"] = idx
Let's look at a few samples.
len(data), data[0:2]
Format our Data for Chat Completion Models
We modify the base_prompt from the LegalBench task to make it a zero-shot prompt, as we are training the model instead of using few-shot prompting
base_prompt_zero_shot = "Identify if the clause provides that all Confidential Information shall be expressly identified by the Disclosing Party. Answer with only `Yes` or `No`"
We now split it into training/validation dataset, lets train on 30 samples and test on the remainder
n_train = 30
n_test = len(data) - n_train
train_messages = []
test_messages = []
for d in data:
prompts = []
prompts.append({"role": "system", "content": base_prompt_zero_shot})
prompts.append({"role": "user", "content": d["text"]})
prompts.append({"role": "assistant", "content": d["answer"]})
if int(d["new_index"]) < n_train:
train_messages.append({'messages': prompts})
else:
test_messages.append({'messages': prompts})
len(train_messages), len(test_messages), n_test, train_messages[5]
Save the data to Weigths & Biases
Save the data in a train and test file first
train_file_path = 'encoded_train_data.jsonl'
with open(train_file_path, 'w') as file:
for item in train_messages:
line = json.dumps(item)
file.write(line + '\n')
test_file_path = 'encoded_test_data.jsonl'
with open(test_file_path, 'w') as file:
for item in test_messages:
line = json.dumps(item)
file.write(line + '\n')
Next, we validate that our training data is in the correct format using a script from the OpenAI fine-tuning documentation
# Next, we specify the data path and open the JSONL file
def openai_validate_data(dataset_path):
data_path = dataset_path
# Load dataset
with open(data_path) as f:
dataset = [json.loads(line) for line in f]
# We can inspect the data quickly by checking the number of examples and the first item
# Initial dataset stats
print("Num examples:", len(dataset))
print("First example:")
for message in dataset[0]["messages"]:
print(message)
# Now that we have a sense of the data, we need to go through all the different examples and check to make sure the formatting is correct and matches the Chat completions message structure
# Format error checks
format_errors = defaultdict(int)
for ex in dataset:
if not isinstance(ex, dict):
format_errors["data_type"] += 1
continue
messages = ex.get("messages", None)
if not messages:
format_errors["missing_messages_list"] += 1
continue
for message in messages:
if "role" not in message or "content" not in message:
format_errors["message_missing_key"] += 1
if any(k not in ("role", "content", "name") for k in message):
format_errors["message_unrecognized_key"] += 1
if message.get("role", None) not in ("system", "user", "assistant"):
format_errors["unrecognized_role"] += 1
content = message.get("content", None)
if not content or not isinstance(content, str):
format_errors["missing_content"] += 1
if not any(message.get("role", None) == "assistant" for message in messages):
format_errors["example_missing_assistant_message"] += 1
if format_errors:
print("Found errors:")
for k, v in format_errors.items():
print(f"{k}: {v}")
else:
print("No errors found")
# Beyond the structure of the message, we also need to ensure that the length does not exceed the 4096 token limit.
# Token counting functions
encoding = tiktoken.get_encoding("cl100k_base")
# not exact!
# simplified from https://github.com/openai/openai-cookbook/blob/main/examples/How_to_count_tokens_with_tiktoken.ipynb
def num_tokens_from_messages(messages, tokens_per_message=3, tokens_per_name=1):
num_tokens = 0
for message in messages:
num_tokens += tokens_per_message
for key, value in message.items():
num_tokens += len(encoding.encode(value))
if key == "name":
num_tokens += tokens_per_name
num_tokens += 3
return num_tokens
def num_assistant_tokens_from_messages(messages):
num_tokens = 0
for message in messages:
if message["role"] == "assistant":
num_tokens += len(encoding.encode(message["content"]))
return num_tokens
def print_distribution(values, name):
print(f"\n#### Distribution of {name}:")
print(f"min / max: {min(values)}, {max(values)}")
print(f"mean / median: {np.mean(values)}, {np.median(values)}")
print(f"p5 / p95: {np.quantile(values, 0.1)}, {np.quantile(values, 0.9)}")
# Last, we can look at the results of the different formatting operations before proceeding with creating a fine-tuning job:
# Warnings and tokens counts
n_missing_system = 0
n_missing_user = 0
n_messages = []
convo_lens = []
assistant_message_lens = []
for ex in dataset:
messages = ex["messages"]
if not any(message["role"] == "system" for message in messages):
n_missing_system += 1
if not any(message["role"] == "user" for message in messages):
n_missing_user += 1
n_messages.append(len(messages))
convo_lens.append(num_tokens_from_messages(messages))
assistant_message_lens.append(num_assistant_tokens_from_messages(messages))
print("Num examples missing system message:", n_missing_system)
print("Num examples missing user message:", n_missing_user)
print_distribution(n_messages, "num_messages_per_example")
print_distribution(convo_lens, "num_total_tokens_per_example")
print_distribution(assistant_message_lens, "num_assistant_tokens_per_example")
n_too_long = sum(l > 4096 for l in convo_lens)
print(f"\n{n_too_long} examples may be over the 4096 token limit, they will be truncated during fine-tuning")
# Pricing and default n_epochs estimate
MAX_TOKENS_PER_EXAMPLE = 4096
MIN_TARGET_EXAMPLES = 100
MAX_TARGET_EXAMPLES = 25000
TARGET_EPOCHS = 3
MIN_EPOCHS = 1
MAX_EPOCHS = 25
n_epochs = TARGET_EPOCHS
n_train_examples = len(dataset)
if n_train_examples * TARGET_EPOCHS < MIN_TARGET_EXAMPLES:
n_epochs = min(MAX_EPOCHS, MIN_TARGET_EXAMPLES // n_train_examples)
elif n_train_examples * TARGET_EPOCHS > MAX_TARGET_EXAMPLES:
n_epochs = max(MIN_EPOCHS, MAX_TARGET_EXAMPLES // n_train_examples)
n_billing_tokens_in_dataset = sum(min(MAX_TOKENS_PER_EXAMPLE, length) for length in convo_lens)
print(f"Dataset has ~{n_billing_tokens_in_dataset} tokens that will be charged for during training")
print(f"By default, you'll train for {n_epochs} epochs on this dataset")
print(f"By default, you'll be charged for ~{n_epochs * n_billing_tokens_in_dataset} tokens")
print("See pricing page to estimate total costs")
Validate train data
openai_validate_data(train_file_path)
Log our data to Weigths & Biases Artifacts for storage and versioning
wandb.init(
project=WANDB_PROJECT,
# entity="prompt-eng",
job_type="log-data",
config = {'n_train': n_train,
'n_valid': n_test})
wandb.log_artifact(train_file_path,
"legalbench-contract_nli_explicit_identification-train",
type="train-data")
wandb.log_artifact(test_file_path,
"legalbench-contract_nli_explicit_identification-test",
type="test-data")
# keep entity (typically your wandb username) for reference of artifact later in this demo
entity = wandb.run.entity
wandb.finish()
Create a fine-tuned model
We'll now use OpenAI API to fine-tune ChatGPT-3.5
Let's first download our training & validation files and save them to a folder called my_data. We will retrieve the latest version of the artifact, but it could also be v0, v1 or any alias we associated with it
wandb.init(project=WANDB_PROJECT,
# entity="prompt-eng",
job_type="finetune")
artifact_train = wandb.use_artifact(
f'{entity}/{WANDB_PROJECT}/legalbench-contract_nli_explicit_identification-train:latest',
type='train-data')
train_file = artifact_train.get_path(train_file_path).download("my_data")
train_file
Then we upload the training data to OpenAI. OpenAi has to process the data, so this will take a few minutes depending on the size of your dataset.
openai_train_file_info = openai.File.create(
file=open(train_file, "rb"),
purpose='fine-tune'
)
# you may need to wait a couple of minutes for OpenAI to process the file
openai_train_file_info
Time to train the model!
Let's define our ChatGPT-3.5 fine-tuning hyper-parameters.
model = 'gpt-3.5-turbo'
n_epochs = 3
openai_ft_job_info = openai.FineTuningJob.create(
training_file=openai_train_file_info["id"],
model=model,
hyperparameters={"n_epochs": n_epochs}
)
ft_job_id = openai_ft_job_info["id"]
openai_ft_job_info
this takes around 5 minutes to train, and you get an email from OpenAI when finished.
Thats it!
Now your model is training on OpenAI's machines. To get the current state of your fine-tuning job, run:
state = openai.FineTuningJob.retrieve(ft_job_id)
state["status"], state["trained_tokens"], state["finished_at"], state["fine_tuned_model"]
Show recent events for our fine-tuning job
openai.FineTuningJob.list_events(id=ft_job_id, limit=5)
We can run a few different fine-tunes with different parameters or even with different datasets.
Log OpenAI fine-tune jobs to Weights & Biases
We can log our fine-tunes with a simple command.
!openai wandb sync --help
Calling openai wandb sync will log all un-synced fine-tuned jobs to W&B
Below we are just logging 1 job, passing:
- our OpenAI key as an environment variable
- the id of the fine-tune job we'd like to log
- the W&B project of where to log it to
See the OpenAI section in the Weights & Biases documentation for full details of the integration
!OPENAI_API_KEY={openai_key} openai wandb sync --id {ft_job_id} --project {WANDB_PROJECT}
wandb.finish()
Our fine-tunes are now successfully synced to Weights & Biases.
![image.png](data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAB20AAAOdCAYAAACmuflRAAAgAElEQVR4Aey9d5QU1f63+1vrXffPu9b9477rvu97fieYMAESZMg55yCIICAIBlAEEUTAAIIIHEyYA0FMiIoKGEBQjwGGIBg4oCISJKpHyQgI37u+NbN7dtdUh+qunq6ueXqtprsr7Nr17KfZNfOZveu//s9//114wgAHcAAHcAAHcAAHcAAHcAAHcAAHcAAHcAAHcAAHcAAHcAAHcAAHcAAH8uPAfwE+P+DhDnccwAEcwAEcwAEcwAEcwAEcwAEcwAEcwAEcwAEcwAEcwAEcwAEcwAF1gNCWkcaMtMYBHMABHMABHMABHMABHMABHMABHMABHMABHMABHMABHMABHMABHMijA4S2eYTPX07wlxM4gAM4gAM4gAM4gAM4gAM4gAM4gAM4gAM4gAM4gAM4gAM4gAM4gAOEtoS2/NUEDuAADuAADuAADuAADuAADuAADuAADuAADuAADuAADuAADuAADuCATwcGVrlEXq9VJDsbNJE/G7dwnvpel+k6P2E8oa1P+H7gsi1/FYEDOIADOIADOIADOIADOIADOIADOIADOIADOIADOIADOIADOIAD0XKg7XlVZFVRA5EmLZM+dRvdNp32J7QltE1LlHRkYpto/YdDe9KeOIADOIADOIADOIADOIADOIADOIADOIADOIADOIADOIADOBDvgI6gPd24RdKw1g5zddt0Rt0S2hLaEtriAA7gAA7gAA7gAA7gAA7gAA7gAA7gAA7gAA7gAA7gAA7gAA7gAA6kcEBHzfoJbE14q/ukGnFLaJsCPn89EP/XA/CABw7gAA7gAA7gAA7gAA7gAA7gAA7gAA7gAA7gAA7gAA7gAA7gQGV0IJ0pkU1Q637VfZMxI7QltE0qSDJ5WMd/yDiAAziAAziAAziAAziAAziAAziAAziAAziAAziAAziAAziAA4XuwN/Pu1iadRkkvW5+UK6Z+JZc/+BqGfbEN85zyANrpO9dS+TevuMST4m8U8oeO99NuF2yaZIJbQltCW1xAAdwAAdwAAdwAAdwAAdwAAdwAAdwAAdwAAdwAAdwAAdwAAdwoNI5cEmtptJt2IMy5tl/y5jZm2XUs9/Krc98J7c+872MfPZ7GfHsVhnx7A8y4tltsm7gBO8w9t3SxPbdliKPbSgJb/V9k/LP12sVJWRMaMsXMKEchf5XEdSfv+zBARzAARzAARzAARzAARzAARzAARzAARzAARzAARzAARzAARzwcqDbkHtl9LObZcxzm+X2Od/K7XO/l1HPfScjntokNz9SLDfO/ESGPVwsI5/aJLc9t01+7dDbM4gVH6HtzgZNEuZyhLaEtgnl8BKYZfzHhgM4gAM4gAM4gAM4gAM4gAM4gAM4gAM4gAM4gAM4gAM4gAM4UKgO1KjbTG5/cIlMfGGzjJ2zRSbM+1bueuF7GTfn33Ld5CXS7bpp0r7nTdKt903Sd8h4GTRihtx45zw507S1d2irI2pNcKvjbBOMstWRt382bpEwl8sotG3b5DzZ/U5N56nvs22UoMvLtj7sz380OIADOIADOIADOIADOIADOIADOIADOIADOIADOIADOIADOIADOBAtB+o16yQT56yXyS9/J5Ne/l7uX/iDPLFku0x+4Wu55vbnpH2PG6R5i87Sqmkb6d6xm/Tp2Ud69ugjva66Vs40beUd2jqB7RGRx1JPjxx4aDtz1MUia2o7T32frbBBl5dtfdg/Wl9A2pP2xAEcwAEcwAEcwAEcwAEcwAEcwAEcwAEcwAEcwAEcwAEcwIHK7UCNus3lztkbZPz8rTL+hR/knpd/lAcWbZPHFn4hQ8bMkk7dBkjb5m2laf1G0rJhE2ndpJk0rldfql96qVxY5ULZ3ai5d2i74YiI7Cxd927JPW13vuu5beDTIwc9Mjbo8jL90p1zXhWpV7+R9LyytzRs3FTOr5J9IJ1pXdivcv/HQfvT/jiAAziAAziAAziAAziAAziAAziAAziAAziAAziAAziAAzgQnAOjHlouk1/dIXe9vEPGv7RT7nz5R7l7znoZdvtD0q3rVdK3xxXSu317aV2/rjSpVUPqVbtUal14gVQ/9xy54C//WxZdVssziC2ZGtk10nbDU57bvl6rKOFg2IymR05XkLCEsanqW79hY5n5wAPyyiuvyMKFC2PPV199VR6ZNUtat22fEGCqslkf3JcJlrD040CPK3rJE08+KQsWLJDZc+bItYOvk7/89R98l7mPNw7gAA7gAA7gAA7gAA7gAA7gAA7gAA7gAA7gAA7gAA5UMgd6DL5HZizaKTPe3CP3vrZb7l64WybM3yLD7pkvPbr2kWuv6iNDr+olt/XuIfcM7Ctj+/aSET26yMhunWVU1y4yrF0bmdmwoWcQq/eq1YG2sUeCUba63cAqlyR0L6ehbdinPdYAZ/CQ65yw9plnnpVrBl4rF11SzYGlo2x79uotjz76mLN+xMhRBD6V7AvsJyBk23AFyn369pPnnntO2rXv6Hyf9Q8zHn/8cbllxMiE/xnShuFqQ9qD9sABHMABHMABHMABHMABHMABHMABHMABHMABHMABHAjCgQsvqSljZr4p9y34Vu5/c49MXrRXJr2+W+546nMZOHiUXNGugwzs2klu7NhKHr9xoLw3daIsm3m/vDpxgjw7aoQ8PnSozBo8WB7of7XsaNk6cXCr4W2S56qiBkkzioxC22qXniOThl3kPPV9ImDpjrRNt7xEx8l0uY6809G1EybcKTo1slc5GuwOu+lmZ7vbRo/x3MZrv2yWaXD8+ONPyJjbxwZ+PC1zwDUDfZfbqXNXmTdvnuhrNucWhX2L6jWQ6dNnxAL+KJxTlM5B/+DiwQcfkr5X949ztVmLVs7yqtVqxC2P0rlzLlzA4AAO4AAO4AAO4AAO4AAO4AAO4AAO4AAO4AAO4AAO4EC8A606XiU3jZ8ldz75oUx+9QeZsmivTF64XW6b/oZ0addZGl5WTW7q3FoeHzpQXht7qywcfYu8MHKYvHz7KFkwfqzMHz1K5owYLo8PGSizu3WSM0mC2USh7enGLaRtgizStJfv0FYD1p+W1hRZU9t56vtkwa05UKLXoMtLdBz38gaNmsjs2XNkwp13pRxBq8HtmDG3y/z58ytkquRchbYaNs6dO9cJHN08Un1OFNpqCPzSSy+VC3M1GLanms5FAJ2qzoajXQ99n21dEp1zqvp4rdeysq2PV7lBLNN62ew0qDblqg/a7vZ62wPd1l5nv/c6X3dbqafqqzmen1ct66GHHi7npC7XemVarp86sG18hwgPeOAADuAADuAADuAADuAADuAADuAADuAADuAADuAADuTDgb+dc74MGDZBrr3lXrnxjkdk1MzFMumFb2Tqy5tl2Lgn5PLqNaTWBefKhN6d5ZU7hsuSu0fL4gm3yfL7Jsqqxx+Rzx57RJZPu1+W3DtRFt4xRh4b3F8WdGonf/oIbjWwTTYtsuHiO7TVEbYmsDWvuswU6Pc16PLSPb5Od/zc7NlyeVG9tOpuQqXJU6akDHnTrUOi7cyxvMKtRPvkerk7tDV11DDODuu0HhrY2qGbe99063pJ1erSvEUr+evfz02rjdzlmjraHE3YmMloY3f5QXzWENGuX7Zl/vffznGYVa2e+WhSw01He+t7Uye7rl5tqudht7vuZ8pKdo5e2+ix9GmO7edVyyO0pfP34wzb4gsO4AAO4AAO4AAO4AAO4AAO4AAO4AAO4AAO4AAORNOBeo1by5ARE2Xg8Lul/7C7ZcAt98nQu56W0dNekUE3TZTLLrlEal94rgxt30Ievq6vLL77Vln1yP2yYsYUmTdquDw0qL88dM3VMrNfbxnftZ1c07S+DGhaX8Y1bSgbmzVLOh2yjrrVKZFTjbA17lXK0FanT33kkUfkrrvv8RUKDb9lhHOfzJq16vjaz8BO99UrxEp331xt5xXS6bESLbfrYc7Hb1CqAWTLVq2lVeu2GQW35rjuwDCbQNA+ryDe20FoEOVpGRdfWk26de8hmU4D7A7dverl1e5mJLfdzonawKtMe5mW4Q6N7fXJ3usxvULbZPuwLpqdMe1Ku+IADuAADuAADuAADuAADuAADuAADuAADuAADuBA5XagW69rZNhtU2TwLffItcMnyuBbJsl1t94nQ0dPlSuvvl4uPv88qXbO36Rng1oyfUBPeXPiKHnq1qFyS9cO0qv+5dKnXi25pVVjuaNjS7m63mVS//y/SpOL/yEtqleRek1ryh1tGsiy5o1lb7PmzujbM01byoHWLeSTbs1kXLu6vvJE36Gtn+mM07mnrZ/ygvpi6ejaOXPm+r636xU9r3RGErZq084XZL/1NkHXvZOnOMczU8vaYZiWqZ/NOn2115vQ6/obhjrbaACmoZq+2iGmhoamDDskM3Uw6zQE87qnrVd45z5frxGY7m0Sfc4muDXnYJ+vHscObXWdCU71XPW9bpOKrXtEqb29e+SxOaZhabeFWaavpp6J2iQRI6/l2QS3hodXuWaZV7sHFdqacgwPc0z7tX3HzjJ//gsxd22O7vePzJoVN2LYLof3lbuzpv1pfxzAARzAARzAARzAARzAARzAARzAARzAARzAARyItgNjxk+SKTOekLunzJIxdz8gt06YITePnSpDb5skPfsMkgvP/Ydc8Jf/T3o2rC2zhl0j0wf2kkHN60vLqhdInX/8RRqf/1cZ3LSeTO7TU7pdXlXO/Z//j9Q7/7+lc60LpW3bOtKyaz1p2KaGNOlQSzpd00CuGtlY+t7WSPqNaSqdr01vtl/joO/QVnfUoFWnNdanvjeFuV9njro4NpWyvnevN5/TLc9sn+2rCYXskDOdMr2CqnT287uNCRvtYNB9bK27HQ7qev1szsmEiCaE1DqYck0YptvYx9DP+jTb2SGu7mMfz5yTu15mub6a8NGug70+3feZBrfmPMz56vHcnHSdHZjqNsrAPlf3Pm5ubgb6+ZlnnondO1XP381Sy9Bj6Tq7fu6y9bPZNl1eZrtMglvDLNUx3eesx9TzsH3SZaY8+xxN/dyvhrPN3r2N+az1G3vHuIT/p5jtzB8q6KtZxmu0O2Dal/bFARzAARzAARzAARzAARzAARzAARzAARzAARzAARwwDjz7/FxZtPhdWfreClm4aKnMe/F1mfHQUzJqzN0ysG9/aXxZVTnnf/+/0rdFA3niliFyc+tG0qlaFWl8wd+l4Xl/k3aXnC9ju7SThePHyoQrukiri86RbjUvlmsa15YruzeS1l2KpKhZVanboqq07VNHrhrZSPqPbSxXj24iHQcW+comMgptzYmmek1npG2qMnKxvtplteSpp54Wne7YT/mDrh1SLpTys3+623oFXe5lGva5w1ANxkw4qKFWqgBNt/EKyLwCOa9lej6Jltvnmug49jap3pcEt22c+7X+5a//SKvdDDP3yEutjzmezcws88vWHbxqObpMj5OKj3vfIFiZ89DXiy+pJt17XCEXXnxp7Jzt9e73hpnNSOtoGOp73UfPS90xy/XV7ZtuZ8pLJ7Q1dTH7mGOZ5far1i+dMglt6Zhtb3iPDziAAziAAziAAziAAziAAziAAziAAziAAziAAzhQuRy4a9oEefS552TBordl6bvLZfE778mjTzwpo24aJhOu6i7DWzWU5lXPl5u6t5enbh4st7VtKtc3rS+Dm9SXIY3ryYiWjeXxa/vLJzOmyou33CSj2jSVsR1byrhubWTAlU2lVefLpW7zqlK3ZVVp0aOmdBxUR3oMrye9bmkgLa+skVY2Y5zMaWhrDhK2Vw39pk2fLg88+KD849wL0gKm+0yeMsW5F67eEzeX52RCK3copSGWPhOt1yDLBGf2e1NXr/3MqGMN3UxI5rVvovAx0XJzTPNq6m4++32tctEl0q1bD6nrY8Sk1/m6j+sObRPtYzOx35vt7fDSvNft7G3dx9bPysXdzl5t4rVvqmUadDds3EQ6d+km555fJW1nveqkx9J6GkfSbXfDxz5HfW8YmT8ycJ9LqvKVq12me3/zmdC2cnW+pt15pd1xAAdwAAdwAAdwAAdwAAdwAAdwAAdwAAdwAAdwAAfUgY5Xt5VBo26R26dMlfsemiX3P/CgjBwzSob0vULu7d5aJndoIsNa1Zd7ruklc2+9Th68ppfM7N9bZvTtJVN6dJIp3TrKvKGDZcWkO+Xl4TfKvV3aydQrOsi0q7vKdVe3kNZdLncC26KWl0qDjtWkcffq0urqmtJp8OXSoMMlaWczWteMQtugR9AGXV46X8Q+ffvJiy++KD179U4LWIdOXWT+/Pmio23TKT+bbbyCLvcyrxBUQywTgnmFhe4y7Dra67wCM69lun+i5XbZ+t6rvu5tEn02ge3ldfzdsNk+p0Rl28zMNl51tbdzs9XtEwWIqfgk2zed+ps6u19NYNuxU+e0/zDBlOE+P7Ncz1Hrq59TnZfZJ9NzSFU+oS2drXGMV1zAARzAARzAARzAARzAARzAARzAARzAARzAARzAARxI5EDDDvXlyqEDZNDoEdJ/+E3S+9oB0rV3d+nTq5NM6N1JpnZtKVN6dZBpg66UuSMHypzhg+Sp6/vLQ317yKROrWVCu+YyvWdnefKaK+XBq7rK2NaN5c42jWRq73Zy/dXNpFXnWlKn2SVSo1EVqdX8Qqnb7hKp3/FSadajmtRqmt7AUVP3jELboO9Vm255ptJBvJ5zXhWZOvV+mT1njjRs3DRpEHtZzdry2GOPy8wHHhDdL4jjJyvDBF0mgNVtNTAzo2j1s4ZW9tTGGnLpZ11u1tvb6zJTrgkYdVuzva43YaXZzgR0Zp19PF2mT69wTcu36+6um9k3nddMA1st25yHOV+v47nrqtv4ZeveXsto0bK1w8fUwebRsVOXuPvd2pwTtYlX3RMtyyaw1TK96qzLjR/63qvdvepjykrWBmZksdtFm5m7bN02WZlme0ba0lEbF3jFBRzAARzAARzAARzAARzAARzAARzAARzAARzAARyofA40aFMkfW68UgbdOkT6DusrVw3pLr0GdJa+/bvJbddeJdP7dZdHB/WS6QN6yKNDrpBZA7vJtB6tZXzr+nJr0yIZ1qiOXFf3MhnesIbc2rxIbmpWJCOb1pa7uzaWgb0aSLMO1aV2k4ukWr3zpVqD86RGk/OlRqPzpUGbS6Vp6wqYHjmdkbHVLj1HflpaU2RNbeep73WZ1xcinfK89st2WdVqNeTRRx+TefPmiY68/evfz42rn06JfGXvPjJnzlxnOlcNbXWfbI+ban8TdN15192x+4Z6BaYaXJlpZvXVDr30farQ9pYRI51tTBl2SGaCNLNOt33mmWecsE7r7z62bmeOp4GlBmpmX32165bq/M3686tc5EyJXLN2nYyYG47Jwj1dZ5+3Obb7/Oz663tzrmZ7E0ybc7bXm3qYdeqb8tR97f00FE3WJuZYyV7V2UaNm0qHjp3l7+eclxE3U767De1wOejQtnmL1gldNPWxX7UNkrWr2ZbQtvJ1wKbteaXtcQAHcAAHcAAHcAAHcAAHcAAHcAAHcAAHcAAHcAAH6jSpLr0Ht5cBw7rL4Fu7yI1jOsmQkR1l0NDuMvSGPnLHNVfItP6dZMrVneWfA7rI+HZ1ZUzzmnJTw+pyQ8NaMqxpXbm5aZHc0qS2jGhWJMOa1ZWbml4u4zrVl369ipzQtmajKlK17nlyaZ1z5aKaf5Pqdc+Tbj0byzVXdfKV02Q00jYdyScNuygW2JrgVpels29FbqOB2r2Tp8irr74qzz//vDOaUEfg3j9tuhPW6nJd3+vKq2T+/Bec0bYVEdxWJIOwHuvc86rIRZdUDZ0zXqFtWBhqaKt+uv8AISz1C6oeV/cbIA8++JBcO3hI0ufNw2+RJ596Si4vqhc6j4JiQTlcdOAADuAADuAADuAADuAADuAADuAADuAADuAADuAADuCAtwOX1LhQuvRvIFfd0EyG3NZGbhzbRm4Y00oGj2gn/a/rIL36NJNBfRvL8F4t5N4rW8uYVrXltmY15eZGNaXf5ZfKkEY1ZXSHhjKua1O5vXNTGVDvMhnerI7c2bOpXNOvkbTqWlMub3Gx1GhaRS6tf65Uqfk3adC6mkwY3U96d+7gK5uo9KGtkbhBoyZy2+gxTsCjIyE16Bk3foLocrNN5y7dCG5Lp0U2TCrjq4a2XqNzKyOLfJ2zhtJX9LwyaWBrAt227fz9p5ivc+K43h0qXOCCAziAAziAAziAAziAAziAAziAAziAAziAAziAAziQqQN//fs/pEu/Iuk2oK5cdX1j6XdzY+k/vJH0v6mxDLyluQy+vaVcN66F3HxdCxnftbGMbV1Lbm9ZWya0bSTj2jSR29s0lTHtm8votk3k1hYNZHTLBjKxWwuZ0K+19BvYTNr2riNNul8mDTpdKnVaXyQ1W1SR9lfUlaem3Crn+7zlakahrU5zrKNm9ZloymM/0yNnCjof+xHcVt7/GOypjNOZmjcffnLMyusnbU/b4wAO4AAO4AAO4AAO4AAO4AAO4AAO4AAO4AAO4AAO4EB5B2o1ulja9qklna+5XLpeW0e6Dy6Sq4bWl2tHN5Ohd7aVm+5uLyNvaSvjujaU8W0ul7Gtasv4dg1kYscWMqVLW5ncpZVM7NRMJnVuKff1bCcTr2wjwwe2lJ4DG0n7/nWkw4DLpVXvGlKvw6VSr/2l0m9gS7n9uqtjg0LTbRPfoa2fMDadcFcrmq972qYLyb0dwW154d2M+AwjHMABHMABHMABHMABHMABHMABHMABHMABHMABHMABHMABHMCBfDvwt/P+Ic2uqC7t+teW9s6zpnQZXFN631TkTJk88s4eMm7UFXJnjyYyvk0dmdDmcpnQvr5M6tZK7u/VSWZc1VWm9e4s9/fqLPf17iy39mguPfsUSdfBdaXL4CLpO6yRXHVDA2nZ8zJp2vkyGXlzd6lXu1buQ9tc3Kt25qiLY/e/1ff5brx0jt+hUxfn/reX1axdEPVN55zYhv84cQAHcAAHcAAHcAAHcAAHcAAHcAAHcAAHcAAHcAAHcAAHcAAHoubAZQ0vktZX15JWfWpKy6uqS5t+VaX9gOrSa0gjuem2HjJp9NUytW97uat9XRnfqpaMa11b7uzQSCb1aCtTr+zkhLdTrmgvY7u1kFYNL5ImnS+RdgNqyRU31JPrR7eUISNbSIc+taRd99rSq0vLjLJD3yNtcxHaFtpI26iJyvnwny8O4AAO4AAO4AAO4AAO4AAO4AAO4AAO4AAO4AAO4AAO4AAO4ECUHajb9hJpekU1aX5lVWnVt6q0uqqqdOhTW24c2klmTxwuswZ2kyldGssEDW2bVZc7mtWQsS0vl/Ft68s9HRvJXV0ayZA2teTCqv9LLm9zvrS6uoZceWNDuX5UaxkwtIm06XmZNGtTI6PAVrn7Dm39TI9MGMuXO8pfbs4Nv3EAB3AAB3AAB3AAB3AAB3AAB3AAB3AAB3AAB3AAB3AAB3CgMBz4x/nnSMMOlzojbdsOuEzaXF1drhhQT2ZOGCifPjZZ5t7YW2b0aCH3tC2S8c0vk3FNqsq4ptXkrtY1ZVLHOnJ31yLp376aXFj7/0hR+wulbb/a0vv6xtJvaDPp2r+ONGh3kZxz/rkVF9qqeOneq7YQpz3mi1UYXyzaiXbCARzAARzAARzAARzAARzAARzAARzAARzAARzAARzAARzAARzw48B5F50nrXvVli6DiqTPdQ1l8rjeUvzcffLt7H/KayMHyEM9W8vkDg3lzpa1ZVzT6jK+WXW5u01Nmdy5SO7qViR9O1WTmk3PkcZdq0rXaxpI7yHNpPvABtKsWzU5p8o5GQe2eg6+R9r6OXFG2vJF8eML2+ILDuAADuAADuAADuAADuAADuAADuAADuAADuAADuAADuAADuBALh248MIqckO/9vL4pCGyfu59svulWfLNY5Pk1eFXy8M9W8v9nRrJPW2KZHyLmjKuxWVyZ5taMqlTXZnQvZ7071kkzbpVldY9a0mPgU2l16AW0rxLbTnngsxH2JpzzWloaw7CK18uHMABHMABHMABHMABHMABHMABHMABHMABHMABHMABHMABHMABHMCBsDgw9dp+sveVWfL9s1OleMYYmTekhzzQralM79RQ7utQT+5pe7nc06623NepSKb3aChT+jSXcSO7y4CbWkuPgQ2le/8mUq9JzaxG19osCG3/my+HLQTv8QEHcAAHcAAHcAAHcAAHcAAHcAAHcAAHcAAHcAAHcAAHcAAHcKAyONCwRg2ZcUM/WTl5pDw9oJPM7NxIpndsIPd3rC9TOtaVKZ2KZFrXevLPKxrKA9e0k2f/OVzunNhXOnRvIBdcfEFgga2yJrQltA1UqMrwBeYc6ahwAAdwAAdwAAdwAAdwAAdwAAdwAAdwAAdwAAdwAAdwAAdwIDoOnHfu+dKuTk0Z0KCmjGpRR+5pW1emdKgr93Uskns71ZXR7YpkQPM60rpxHfn7OdnduzaRN4S2hLaEtjiAAziAAziAAziAAziAAziAAziAAziAAziAAziAAziAAziAAziAA3l0gNA2j/ATJeksj85fZtCWtCUO4AAO4AAO4AAO4AAO4AAO4AAO4AAO4AAO4AAO4AAO4AAO4EAqBwhtCW35qwkcwAEcwAEcwAEcwAEcwAEcwAEcwAEcwAEcwAEcwAEcwAEcwAEcwIE8OkBom0f4qRJ11vNXFziAAziAAziAAziAAziAAziAAziAAziAAziAAziAAziAAziAAzgQfQcIbQlt+asJHMABHMABHMABHMABHMABHMABHMABHMABHMABHMABHMABHMABHMCBPDpAaJtH+PxVRPT/KoI2po1xAAdwAAdwAAdwAAdwAAdwAAdwAAdwAAdwAAdwAAdwAAdwAAdSOUBoS2jLX03gAA7gAA7gAA7gAA7gAA7gAA7gAA7gAA7gAA7gALjKhzAAACAASURBVA7gAA7gAA7gAA7k0QFC2zzCT5Wos56/usABHMABHMABHMABHMABHMABHMABHMABHMABHMABHMABHMABHMCB6DtAaEtoy19N4AAO4AAO4AAO4AAO4AAO4AAO4AAO4AAO4AAO4AAO4AAO4AAO4AAO5NEBQts8wuevIqL/VxG0MW2MAziAAziAAziAAziAAziAAziAAziAAziAAziAAziAAziAAziQyoH/kog8vv7664icCacBAQhAAAI2Af3//eTJkzxhgAM4gAM4gAM4gAM4gAM4gAM4gAM4gAM4gAM4gAM4EFkHCG3tZID3EIAABCAQOgKEtgTWhPY4gAM4gAM4gAM4gAM4gAM4gAM4gAM4gAM4gAM4EHUHCG1DF09QIQhAAAIQsAkQ2nIxFvWLMc4Px3EAB3AAB3AAB3AAB3AAB3AAB3AAB3AAB3AABwht7WSA9xCAAAQgEDoChLZcrHDBigM4gAM4gAM4gAM4gAM4gAM4gAM4gAM4gAM4gANRd4DQNnTxBBWCAAQgAAGbAKEtF2NRvxjj/HAcB3AAB3AAB3AAB3AAB3AAB3AAB3AAB3AAB3CA0NZOBngPAQhAAAKhI0Boy8UKF6w4gAM4gAM4gAM4gAM4gAM4gAM4gAM4gAM4gAM4EHUHCG1DF09QIQhAAAIQsAkQ2nIxFvWLMc4Px3EAB3AAB3AAB3AAB3AAB3AAB3AAB3AAB3AABwht7WSA9xCAAAQgEDoChLZcrHDBigM4gAM4gAM4gAM4gAM4gAM4gAM4gAM4gAM4gANRd4DQNnTxBBWCAAQgAAGbAKEtF2NRvxjj/HAcB3AAB3AAB3AAB3AAB3AAB3AAB3AAB3AAB3CA0NZOBngPAQhAAAKhI0Boy8UKF6w4gAM4gAM4gAM4gAM4gAM4gAM4gAM4gAM4gAM4EHUHCG1DF09QIQhAAAIQsAkQ2nIxFvWLMc4Px3EAB3AAB3AAB3AAB3AAB3AAB3AAB3AAB3AABwht7WSA9xCAAAQgEDoChLZcrHDBigM4gAM4gAM4gAM4gAM4gAM4gAM4gAM4gAM4gANRd4DQNnTxBBWCAAQgAAGbAKEtF2NRvxjj/HAcB3AAB3AAB3AAB3AAB3AAB3AAB3AAB3AAB3CA0NZOBngPAQhAAAKhI0Boy8UKF6w4gAM4gAM4gAM4gAM4gAM4gAM4gAM4gAM4gAM4EHUHCG1DF09QIQhAAAIQsAkQ2nIxFvWLMc4Px3EAB3AAB3AAB3AAB3AAB3AAB3AAB3AAB3AABwIPbf/44w/57rvvZNmyZfLqq6/KnDlznKe+12W6TrcJ+qG/1OcBAQhAAALRI0Boy8UKF6w4gAM4gAM4gAM4gAM4gAM4gAM4gAM4gAM4gAM4EHUHAgtt//zzT9mwYYPMnz9fVqxYIVu3bpWDBw/KqVOnnKe+12W6TrfRbXWfoB6EtkGRpBwIQAAC4SIQltD22LFj8uuvv8q+fftk7969PGGAAziAAziAAziAAziAAziAAziAAzgQOQf09x76+w/9PUjUwxHOr3IGgF9t/10GPvWN1BpfLNXvWM0TBo4L6oS6ke//FwIJbY8ePSqLFi2S5cuXy6FDh1L+tl+30W11H903iAehbRAUKQMCEIBA+AiEIbTVH1T0hxZ9PXv2bPggUSMIQAACEIAABCAAAQhAAAIQgAAEIBAAAf29h/17kHwHGBy/cgaruWp3DeUIawmqE4X16ka+g9usQ1sNXV966SX56quvfHcJuo/uG0RwS2jrGz87QAACECgIAmEIbc1fmBYEMCoJAQhAAAIQgAAEIAABCEAAAhCAAASyJKDBrf4+JFfhGeUSxubDAR1NmSiwYzlhrjqgjuTDTXPMrEJbnd5YR8tmEtiaPkP31TKynSqZ0NYQ5RUCEIBAtAiEIbTVUbaMsI2WV5wNBCAAAQhAAAIQgAAEIAABCEAAAokJ6O9B9PchJkjglZA1Cg4wypZgNlU4r47k0/WsQtuNGzc60xwn/q9d5Pjx4/Laa6/Jtm3bEm6mUyXrPW6zeRDaZkOPfSEAAQiEl0AYQlu9hy0PCEAAAhCAAAQgAAEIQAACEIAABCBQmQjo70PyGV5wbILioB1IFdixnlBXHQjaOz/lZRza/vHHHzJ//vyU97A9ePCgzJ49W3744YeE/Zne41bL0jIzfRDaZkqO/SAAAQiEmwChbbjbh9pBAAIQgAAEIAABCEAAAhCAAAQgEE0ChLaEpn7CpkLYllCWUDYdB/Lpcsah7XfffScrVqxIqzdKZ0pJLUvLzPRBaJspOfaDAAQgEG4ChLbhbh9qBwEIQAACEIAABCAAAQhAAAIQgEA0CRDaEtrmM7zKxbHTCezYhmA3F+6lW2bGoa1Oabx169akvdFvv/0mOoXy2rVrk26nK7WsZcuWpdwu0QaEtiVkjh49Kg/Pelx+SDIddSKGqZarVC++8kpOyk51bNZDAAKVlwChbeVte84cAhCAAAQgAAEIQAACEIAABCAAgfwRILQltE03aCqU7QhkCWTTcSCfPmcc2r766quiUx8neui9bNesWSMLFiyQVatWJdostlzL0jIzfRDalpBbu3adLH33vUwxptxPw2ANhTUc5gEBCBQmgdN//ikHDx2S3w8ezOh55MgRSWcGhaDoENoGRZJyIAABCEAAAhCAAAQgAAEIQAACEIBA+gQIbQlt8xle5eLY6QR2bEOwmwv30i0z49B2zpw5cvr06ZT/w7/++uspR+RqIVqWlpnpI6jQdv/+AzLl/uly25g7yj1zGYZmet72fslG2aoQOkpWz0uD3UQPU4Z9/vb2phx7WaKyWA4BCISPwIkTJ+ShWY/LsFtuzfh504hR8v7yDyrs5AhtKww1B4IABCAAAQhAAAIQgAAEIAABCEAAAjEChLaEtukGTe7tDh06JP/5z39EX93r8vmZQJZANh0H8uloVqHtqVOnYv+Be73RIPbZZ59NOiLX7KdlhSm0dYeSJszU4FMbLIwPrbNX/XR5ohDWPg8TWNvhtC575NHHRV/NI9FxzHpeIQCB8BLQ0bUzH57ljLA1tdz45Vcye+7z5mPKV7/bpywwxQaEtikAsRoCEIAABCAAAQhAAAIQgAAEIAABCOSAAKFt+EPbb7/91hk0Z0KmXbt2ObesNJ/z8bp582ZnVtX333/fedXP+aiH1zHTCezYhmDXy52KWpZxaJtqemTtI/bv3y/z5s1LaxrNsEyPbIJLd2ir56NTA2v4mYv7xWbbp6owGth61duUnezczP52YGv2c796BbnubfgMAQiEk0AuQtt9+/bJhx9+KL///ntOTprQNidYKRQCEIAABCAAAQhAAAIQgAAEIAABCCQlQGgb/tBWQ9qFCxfKN998Iz/++KMTkm7ZsiWvIemyZcvE1EFDZf1cUYFXquMQyBLIpuNAKo9yuT7j0Fa/aFu3bk36n/qmTZvknXfeSbqNWallaZmZPoKeHtkr/HSHnmb0rTvo1AbTANVebpZpuWY/M/rVa3SsOZbZRqds1mWJHqbMZIGyKTPZuSXb3xw7nWOZbXmFAATCRSAXoa3e31bvYf7uu+/mJLiNSmj7n99+k3998qnMmfe8PDt7jjw3Z54sfedd+Wn37rT+uClcJlEbCEAAAhCAAAQgAAEIQAACEIAABKJOoLKGthqAahCqA9fcT12+cePG0ISQmjtocKv1fOWVV+T777+vsLodP35cVq5cKfPnz5fXXntN9uzZI4cPH5alS5fKjh07nHroMv2doS7X97qdbq/76f65DL+8yk4nsGMbgl0vdypqWcah7XfffScrVqxI2i999NFH8vnnn8vPP//s/EeWbGMtS8vM9FERoa0ZaWtCTxNe2uGs1l8bL1FoqyHsw7Med4Jb3dZdpi7zCld1O69w1/DSfdzTGJt15tWrXLNOz8mul1nu9WrOz3Dw2oZlEIBAOAloaHvnPfdmfD9bcy9c93TKuQxuCz20PXPmjGzZ8q0sfP0N2frDNvmjdIr9P//8U/bs2SuLl77jhLmpbjkQTqOoFQQgAAEIQAACEIAABCAAAQhAAAJRJZCL0FbDO80N3njjDSdsNEFIouVmfUW9/vbbb/LWW285oaPek9X91DBSA9Lly5eLbltR9Up2HA1qNbB9+eWXKzRQ1kEcGtDqLKpffvmlc3wNYxctWhRjo+vefPNNJ6TV+ul2ukz30/2TnVcu1hHIEsim40Au3Eu3zIxD2z/++MP5iwi9kXSih85Zrve01f+E9ZfWiR5ahv51hZaZ6SPXoa0JPO1gM5PQ1h28akPpMnu5hqGpRta6OWmoa9fNvV4/m3PwCls1eNY66Dnpqxnha9fLLlO3d4fV9nreQwAC4SSQi5G25kxzFdwWemj7/fdb5Z1334v9sY7hZV41vC1es1bWf7EhaV9ptucVAhCAAAQgAAEIQAACEIAABCAAAQhUBIFchLYff/yxM/JSQ8b33nvPGW2pIy41BH377bflhRdekLVr11Z4mGcClWPHjjmzh2r93KNszWjWJUuWOPXVWUZ1e7NvPl51GuIFCxY4UyPrSNbXX39d1q9fXyF10plTdbZVc97qy6effipub/TzqlWr4pbrfvmYNjmdwI5tCHaN0/l4zTi01U5BpwHQ/0wTPU6fPi1HjhxJtDq2XL+cGzZsiH3O5E3Qoa0JLe1Xd5AaRGir56rhpx24mtG3fkLRIEJbHalrj9Y15+cV3BLaZmIp+0Ag/wRyGdrq2WlwW1xcLPpHO3rBHcSjkENbvXB/9/1lolMjJ3voHy0tW/5B0mnwk+1ftm6zrFy40JnCR6frWbhwsRTvL1tb4e/2F8vifNchg5PevHKhLFy5OYM92QUCEIAABCAAAQhAAAIQgAAEIBAdAu7wLdsAY//+/c5ISy1n9erVTiiqI1m/+OIL5/OJEyecW2/lY9pc+9z0+DqK1j3KVj/rcv09zoEDB5wRubrM3rei32/btk1++umnWB1+/fXX2P1kc12XzZs3OyGxzrTq51i6vYbLur+f/YLYtmID2R2y5kT8/wfb1hOIVmwbZMY7CNcyLSOr0FZHB+lQ96+++irePB+fdF8tQ8vK5hF0aGtGoypYDS29AlQTarrXee1jliUKQO3QVjno8U1g7LWPm1UQoa07lNZjmABZX+2HnnM69bL34T0EIJB/ArkObX///XfnryX1r+r0D3eCeBRyaPvjj9vl81Wr0xpBq//P6ojbZDNTJOdZEtguXmPd/3zLSlm4cKXkLX4ktE3eZKyFAAQgAAEIQAACEIAABCAAAQiEmEDQoa0dYvz444/OdLo6qjZf9ze16+P3vYa1OjI436Gt33oHvb0O3vDTfhqI660y9XeHQdclnfIqLjA8IE6i8vMBKTumLjsta5ZmFiSWlZN4/5nbTovEHTPxtumUl/9tSoLvig6703EpV9tkFdpqf6LB5UsvvZRRcKuBre6rZWT7yFVoq/VKFFzmMrQ1POzw1h0Om21MHd3Br71e36eaHtlr/0T7aF2S1cd9bD5DAALhIKCh7V2TJsu4u+6RCXdPcp63j7tTRo4eG/tslid61e3d97TVszOBrU53ElRgq+UWcmi7/osv5Ns079f+2++/yyeffpY5u3wHtF6KE9p6UWEZBCAAAQhAAAIQgAAEIAABCECgIAjkMrTV+5rqfW11pjC/IzVzFZb88ssvzn1W9Z62XlMj28t0+mSte2UObfW2l+vWrROdLlpnXDXtoqN9NZhVXvqqn806DW115lXdT/c3yyvqtaJCyEU/S17CU0LbYELqivLR6zhZh7bau2h4qaNldarkZPe4NT2RbqPb6j5BBLZabi5DWwWno0rdI0srIrQ1zDQg9RoJa9ZruGpPbWyW26+JAljdRoNpr/K99jE8zGhk+xi8hwAEwk1AQ9t/Pviw7Nz1k+h7fX62arU89cxzsc9meaJX3d4d2uYqsFWahRzarlm7Lu3Q9vffD8rKDz+SP06ezEwiJyBdKCu3pNi9dLuS6ZNd27vWLVxcLNa4XdFpg3UkrzN9sE6/bKYQdu8Xt1ynaLanbU4yZbNTzkrZ7CrPfU6x45dOBR23XsNrrXdcGeVHGx9Ys9iaRjp+fbnpkZ1AvGza6bjjpcDNaghAAAIQgAAEIAABCEAAAhCAQKESyGVoq+Gd3tNWg1sNcL3Ci4papr/X0iBRg1i95+7u3bs9p0bWgNY8d+3aFYrpkSuKkX0cDVxffPFFh5e2n7Iw6zWg1dD7yy+/dFjpq362g1vdXvdT3lqOTo9t9s/1a8WEtiWjbFONDtVg9/C2HeIEvPqfhBkhu/SQHLb+04gvp3QEb2z9cVl0hwaV5adilhOHZKazbrU4YW65fTwCzvXHxdlPX52HKT/+uFrvGEunvsdlUdJ6r5bqrvVxZdyx2uEQx+PIyTgOWh33PrE6lJ5nUJ9z7WGy8gMJbRWWTm+s96WdP3++89cTW7dudf6z1dFW+tT/eHWZ/mWFbqPbZjslcok0Jf/mMrTVI5jRtnZQqWC9wlwTdNojURNtq2Xrdl6jXO3zSxSqmm1MgKzbJXqYetnnYLY1+7vXeR3XbJvsWKZcXiEAgXAR0CB25sOznIDW1Gzjl1+VC2HNOq9X9/b6hzh6ka3ToWQ+ta/XkUqWFXJou+Xb72Tjl18mPjlrzd59++TTzz7Pqm+MhZmusDV2GCfItEPTkjC1JITU93Z4WX66ZVN+XGhZGo6WTcusoe7KknvpxoLTsnKdMpLWb2FJ6Fpa6ZJwtazO+rnsWCIl68vKFxOwxo5xQIoXWwGzlO4TWy9Ssk9ZGfGhrc1IK+XmFKPLGwhAAAIQgAAEIAABCEAAAhCAQKQI5DK03bRpk3OLLQ3u9uzZU2GhnTss0WPrH7ZrWHvs2LFYPfS93ivWLHN/1vC2Mk6PrOetIey+fftirGymGsC6pz7Wz4mCWS1n6dKlFRbcBxXqJS3HCTxTT4Nswtq4UNYJNu197QBYg1l7XWlQa8Le0uAzFv7aga0V4FZ36mfCWFdwa8Jaq8zqd9h1KAuIYwGqCWOtY5SExFZdS7cpO9eSMmNlmLqLSNk2eiz3sV31DTisNe1qO13R7wMLbU1vpDfh/u6775y/TNHh73PmzHGe+l7/WkXX6TZBP3Id2mrDaEDrDlfdI2A10DRBbqahre5nj+o1x7aXufmZbdyhq71dstBWt9N99T66Jow14ax9HrqdlpNqVK99XN5DAALhIZCr0Hbbtm1y9uzZnJxoIYe2eiG7fMXKlP2eht2rVhfH/v/NCqQJLZ1RqGVBpJapYaQdeOoyJ/Q0I2NdB44PL0v2j42uLd3WvU1cEeVC4vIBacrtpSR0ddc7tp/7GM75x5+3c46xkFZD17IQuKSckmOYMDrunNzlxw7MGwhAAAIQgAAEIAABCEAAAhCAQLQJ5Cq01emQdXCXlq+5gY7G1N9v6+hb/R2TvurnXD+1Hq+//rps3Lgx7lg6QEEDWR0NqgMVdECaGRWsy3V9ZQ9t7ZGzdjutXr1a9JlqmVmv5SjT3377LW4fsz7oVxPK5fQ1WShqhYxeUyib0bd2/ZJNeeyss8LS8mVq6GmFp87xk9wn1qPunsfX7cxxywXN5YPd8vVaLe7w2HMbQtvC7WRyHdoqGQ0zNdS0w1P9T0M/63J9mlBXg0477DTb2fsa2rqd2U+DUv0LHrtM9zHNfu5XDV3d5Zug1tTPfvWaDtmco9nOKwT2Oo67LnyGAATCSeDw4SMyeer0rO9p+/KChRV2goUc2powdv0XG5KOQt6zZ68s+2BFynDXF/Ryo1xLR5yWTilspkd2Xu3QNrZf6XTAsbDTK7T1GajqCXiEqrHz8gxIS+tt17E0gLbPwQSuXuXHhbbu87N4mDLiQlsTbHuE4LF68wYCEIAABCAAAQhAAAIQgAAEIBBBAkGGtmvWrJEFCxbIjz/+6ASgO3bscMJZDUN15KaOeNXRrt98802FhHcaDGtw7A4Y9ff4GiDqrR3Xrl0bq5vWUT/rcl1f0aGt3jNWj+sn0NbRwbqPDqILKvzUMk2b6ax7P/zwQ2w0sh5D21cZ7d+/3zmmvupnXW7qoGXofrq/cl25cqWv8zLlZPJqh6E5e+8En+6gtPwI0fIhpccUx+b/FY+Rr2aVSNmo2XJlmlGwZRvH3sWPaC2tn0do65QZ28t6k0ZoWzLqt+S87FG1DntX2Fuu7k7AzEhbi3hhvQ0qtC2ss46vrRkZa0bKxq8N5pP+R6jBsFeYG8wRKAUCEMg1AR1t+8WGjbJ2/RcZPTf9e7NzkZXrepryCzm01XM4pvdoWbZc9P62ersA+/HnmTPOPW+ff+FFWV28Jqupke1yy97bU/va78u2KHtnQt2yUahxYacJSuPC05Iy0x4FqwfLMLSNHcOMJI7Vw3VeHuXHnYfH+jIGJe/coa1Z75Sj4a0VZJt1vEIAAhCAAAQgAAEIQAACEIAABKJGIMjQVqfH1dsmvvzyy6JTI5vAbfPmzfLCCy84y7ds2RJbbtbn6lVDRB3hmeh+urr+ww8/jIWN7s8VGdrqaFQd9aujgtMNOM0+r732mnz00UeBctXgWEN3vbftkiVLZPny5XHl68hpnfZaQ3p9NSOptS11X52RVffT/bWcIEPlVL7kLKi1RtCae7d6hqLWduVDytQBpQlQY2W7QtZyZbrWpzz/ctsnCFyt8yg5X3dIbe9Xcl7eoW3ZdMjl6k5oW9hdCqFtSftpmGqP8A26VTUQNqOCgy6b8iAAAQh4ESj00FbPSS8Yi9eslRdffkU++vhfoiNvP/t8lfNZp0/WC31drtsEeb/3kvuvlk2JnCiMdLh7hJlxYadnaOs1+tZqRa+Rsx7Hie3htb1zD9mFUjIK1mtkr8/QNq682JHj3iTl5FnHuN35AAEIQAACEIAABCAAAQhAAAIQiASBIENb/d2IjhbVpztY0+mG/Ywgde+fyWe9z6qGhpnsq/tUZGiro5B1tKqORH733XfjRrYmqv+uXbucoNfPPonKSrZcb4epAay7XbU9vUYG63a6ve6XrNxcrUsZWtphZBbvvQPI+NG2Xtt4LYvV2TUy1VnuClnL7586CI6Vr+frKk/Xuadgjtte9/Gql2ta4/L1Kn8sz21c5ZQ7dhZtlKysXPmXTrmB39M2X70RoW0J+VyOtlWhdJRtLkfy5ssfjgsBCISXQBRCW0P35MlTsnffPtmxc6fs3bvPucguW3cyq+DWCRoXetzL1V5WOjVwbNSqHnzLZtnsvK6Uhfa9Xp1wNX5UqWeY6VHm5jXFckDL9Ao4U4a2C8W+b65zzNjIVvdUyWZ0sAl1vUfyusPnkhGzZSOKtaqbtzgU9K1z799YHfQcYqN6vct3duIfCEAAAhCAAAQgAAEIQAACEIBAxAgEHdqmE1hUxDYaHC5evNgZBar31M3kqSNI9bZNGt5WRJ31vrsaduqtFdM9nk41vXTpUl/7pFu2bqf3BH7zzTdl+/btaddJ99PtdT/d38/xgtg2WVAX7LqSsLRkemAT1sYHqJ4hZel0xnGjUtcfkEWxcLRsZKoZ0WtPj+wVsDrLXPe1XbT+gHier0doW700OI07l6UHZNHS0vMqrbO93jk3M32yV91Ly7TP05PHHfaIXcMx969BuJZpGYS2EetIOR0IQAACUSMQpdA2VdtoZ57piNsD+0Vi0/fG7tMaH+I6xy8NWWP3g128WIq3lNSsJPgtvZetBpUasMYCU1eYaZ9MuTJXyub9mYa2i6V4jQbIpfWwQ2c9ZtyxNHgtCW7N/Wi9pl92h7ZajJvV4sXFJeG1tU7D7QNbNkvxSlMXfY0Pe20MvIcABCAAAQhAAAIQgAAEIAABCESJQD5DW50y9/vvvxcdhZtp+JFoP50SWUNbnZpXQ9dMnlu3bnVGslZUaJvoXPK5/IMPPoi7J7C2WaLppnW5PQ2y3ktY96/o+nsGlTkarVm9NHC0/0/QaY1nLt3hBKbeIaUZuWrtdeK0rFlfElSWBLCl6zQUdQLTsnvaxgJWZ5Oy5XH7icjhE4dKgmD3uXuGtnrs0hA6Vq3Tsm19yXnERtpuOx5bawfJMeYm3C3dyg5sdZuEPJw6le4Ud2/f3IW3Fe2lfTxCW0sj3kIAAhCAQPgIVKbQVulrJ/3xvz6R3Xv2hK8xcl0jJ5AlFM01ZsqHAAQgAAEIQAACEIAABCAAAQikQyCfoa1Or6tT++ro0qCDWw1a9Y/FdbRsJqNsdZ8XX3xRiouLcxY6ah23bduWcNpoDUF1/bFjx3JWBztI8nqv9/zV8FXX6T10ta30nsU6utdMl6z1e+edd5zlul630+11P93fq9xcLosFiO6wks/eI2/T4eKEse572uYuUK2INsylg6nKJrRNp3diGwhAAAIQyBuByhba5g10GA5MaBuGVqAOEIAABCAAAQhAAAIQgAAEIAABh0C2oe1HH32UcSiqwaiGqvPnz3fCQBMCpgo80lmvIz71fquZjLA1+wQdJNv11rq98cYbTtC5YsWKcsGtBp+6XvkoY3vfinyv0xu//vrrTvssWLBAdApnrftbb73ljGLWupgwV5fret1Ow1vdL9rTIxd2aOkrGCW0DfQ7SGhLBwwBCEAAAqEmQGgb6uYJtnKEtsHypDQIQAACEIAABCAAAQhAAAIQgEAWBLINbTXYNCFnJq9ffvmlE/LpNMkVGUbm+1h79uyRt99+W/71r395Bta7du1yQttPP/1U3nvvvbyOttU23rJli+zfv99pIx0hvWzZMmcUsHLU0cB2HXU73T6XoXey9vMVRqYzypRtJDY9srnHbQSYJHMo1+sIbbPotNgVAhCAAARyT4DQNveMOQIEIAABCEAAAhCAAAQgAAEIQAACEHATyDa0zSbc0JG1Gv79+OOPlSqwNcw2b97sTB+8b98+z/NPtd6Uk4/Xzz77zAlqN23a5Lzq53zUw+uYhLaVzTkHtAAAIABJREFUaARwFuGxlzsVtYzQ1t0T8RkCEIAABEJFgNA2VM1BZSAAAQhAAAIQgAAEIAABCEAAAhCoJATyGdpWVEDCcU4GHqgePnzYGSWs9w3W0cL6OSycCW0JbdNxIJ++EtpWkg6W04QABCBQqAQIbQu15ag3BCAAAQhAAAIQgAAEIAABCEAAAoVMgNA2+EAzn2EQxz4p6QR2bEOwm8/vCqFtIfea1B0CEIBAJSBAaFsJGplThAAEIAABCEAAAhCAAAQgAAEIQCB0BAhtCW3zGV7l4tgEsgSy6TiQC/fSLZPQNnRdIRWCAAQgAAGbAKGtTYP3EIAABCAAAQhAAAIQgAAEIAABCECgYggQ2hLaphs0Fcp26QR2bEOwm0+fCW0rpn/jKBCAAAQgkCEBQtsMwbEbBCAAAQhAAAIQgAAEIAABCEAAAhDIggChLaFtPsOrXBybQJZANh0HcuFeumUS2mbRabErBCAAAQjkngChbe4ZcwQIQAACEIAABCAAAQhAAAIQgAAEIOAmQGhLaJtu0FQo26UT2LENwW4+ff6vKAno7lT4DAEIQAAChU8gDKHtvn375OzZs4UPkzOAAAQgAAEIQAACEIAABCAAAQhAAAJpENDfg+jvQ/IZXnBsQuOgHag1vliilIlxLsEHzOpI0N75KY/QNo0Oik0gAAEIQCB/BMIQ2v76669y7Nix/EHgyBCAAAQgAAEIQAACEIAABCAAAQhAoAIJ6O9B9PchfsIGtiVkDbsDA5/6htD2juCDziiFx+pIPj0mtK3Ajo5DQQACEICAfwJhCG31BxX961J9ZcSt/zZkDwhAAAIQgAAEIAABCEAAAhCAAAQKg4D+3sP+PUg+wwuOTQgctANfbf9dGG1LaJsoZFY31JGgvfNTHqFtYfSV1BICEIBApSUQhtBWO1bzF6Ya3uo9XXjCAAdwAAdwAAdwAAdwAAdwAAdwAAdwIGoO6O89zIxjfoIGtiVgLRQHNJTT0ZSEt4S3JrxVF9SJfAe2+h0itK20MQgnDgEIQKAwCIQltC2UC0/qyQ9JOIADOIADOIADOIADOIADOIADOIADOIADOIADOFB4DhDaFkZmQS0hAAEIVFoChLaFd3HBBSFthgM4gAM4gAM4gAM4gAM4gAM4gAM4gAM4gAM4gAP+HCC0dcUgx48flx07dsimTZtEgwKeMMCBzB3Q75F+n/R7xQMCmRIgtPXXsXMhBC8cwAEcwAEcwAEcwAEcwAEcwAEcwAEcwAEcwAEcKDwHCG2tFEGDJQ2ZdM7+M2fOWGt4CwEIZEJAv0f6fdLvFcFtJgTZRwkQ2hbexQUXhLQZDuAADuAADuAADuAADuAADuAADuAADuAADuAADvhzgNDWykR0RKAGTDwgAIFgCej3Sr9fPCCQCQFCW38dOxdC8MIBHMABHMABHMABHMABHMABHMABHMABHMABHMCBwnOA0NZKEHQ0ICNsLSC8hUBABPR7pd8vHhDIhAChbeFdXHBBSJvhAA7gAA7gAA7gAA7gAA7gAA7gAA7gAA7gAA7ggD8HCG2tBEGDAR4QgEBuCPD9yg3XylAqoa2/jp0LIXjhAA7gAA7gAA7gAA7gAA7gAA7gAA7gAA7gAA7gQOE5QGhrJR6EShYM3kIgYAJ8vwIGWomKI7QtvIsLLghpMxzAARzAARzAARzAARzAARzAARzAARzAARzAARzw5wChrRV8ECpZMHgLgYAJ8P0KGGglKo7Q1l/HzoUQvHAAB3AAB3AAB3AAB3AAB3AAB3AAB3AAB3AAB3Cg8BwgtLWCj0xCJb1X56HDh2X7jh3y+8GDcvr0aatE3kIAAoZAJt8vsy+vlZsAoW3hXVxwQUib4QAO4AAO4AAO4AAO4AAO4AAO4AAO4AAO4AAO4IA/BwhtrSzET6h06NBheemVBTL05hEyYNCQ2HPw9TfKc3Pmyc8//2yVzFsIQMDP9wtaELAJENr669i5EIIXDuAADuAADuAADuAADuAADuAADuAADuAADuAADhSeA4S2VjKQTqh09uxZ2fTvzTJ85Ch54OFHZMu334qKrw8dZbtjx055dvYcuXnEKCles1Z0ex4QiBKBP/48IyfPnPF9Sul8v3wXyg6VggChbeFdXHBBSJvhAA7gAA7gAA7gAA7gAA7gAA7gAA7gAA7gAA7ggD8HCG2tyCOdUEkDWw1kV6z8UHRqZK+HBrUbNmx0tvvs81Vem7AMAgVLYMDH6+TeDZt91z+d75fvQtmhUhAgtPXXsXMhBC8cwAEcwAEcwAEcwAEcwAEcwAEcwAEcwAEcwAEcKDwH8hraDpu7RfYdPOk89X31O1Zn9cw2vUgVKh08eEimTpshy1esTDmCVoPbTz79VCbcPVEOHDiQbdXYHwJ5I6Cjap/7drvsOHLMqUOfD9fEQtuv/3NQXv5hl6QznjzV9ytvJ8iBQ0+A0LbwLi64IKTNcAAHcAAHcAAHcAAHcAAHcAAHcAAHcAAHcAAHcMCfA3kNbTWwHfDkJuep78Me2up0x9NmzJSjR4+mFXKojE8+/YwsXrI0ZcibVoFsBIE8ENDQdvKGLfI/X1jivHZdvkrGr9skE9Ztkv/14lIntE2nWoS26VBiGy8ChLb+OnYuhOCFAziAAziAAziAAziAAziAAziAAziAAziAAziAA4XnQF5D2/0FFNrq/WrnzJsvy5av8MoUEi5bu269TJ/5gBw9VjJKMeGGrIBAyAnoqNrL31wp/+O5N+T/mr1IOi/7PDb6Np2qE9qmQ4ltvAgQ2hbexQUXhLQZDuAADuAADuAADuAADuAADuAADuAADuAADuAADvhzIK+h7bpth+TIiT8LYnpkHV07Zeo02fLtd16ZQsJlO3bukkmT75Nffvkl4TasgEAhEPj1j5PS/YNVTmD7f897yxl1e+z0n2lXndA2bVRs6CJAaOuvY+dCCF44gAM4gAM4gAM4gAM4gAM4gAM4gAM4gAM4gAM4UHgOBB7adpixQTb9dFQOnzgts5btSjjl8f2Lt8vXu45IzfHFCbfxO12y6/f8vj8mC5UOHznihK9+Q9s9e/bKxMlTRF95QKAQCfx59qzM+36HMxXyjZ9tkE7LPpfbir+SIZ98IVUWvi8r9qR3z+Zk369C5EKdK44AoW3hXVxwQUib4QAO4AAO4AAO4AAO4AAO4AAO4AAO4AAO4AAO4IA/BwIPbZds+EUeeX+XNJ+yXg4cOikd/7mxXCjb8+Gv5D9HTokGvH6D2WTbZxshJAuVTpw4IQ/Pekx0umM/j03/3uyEvRr68oBAIRI4dOqUTNqwWXR6ZH30+XCN3Lths5wVkZV7DsiTm7c571OdW7LvV6p9WV+5CRDa+uvYuRCCFw7gAA7gAA7gAA7gAA7gAA7gAA7gAA7gAA7gAA4UngOBh7ZLN8aHtjOW7JDLxq2OhbNFd62RbQeOyx0LtsaWJQti/azLNtZIFiqdPXtWFr6+SF548WU5c+ZMWofSfRYvWercC1fvicsDAlEgYEJbv+eS7Pvltyy2r1wECG0L7+KCC0LaDAdwAAdwAAdwAAdwAAdwAAdwAAdwAAdwAAdwAAf8ORB8aLvhF/n1yCk5dPy0vPjZPtn00xHnqe912S+HT8k7G38JPLDVcDfbR6pQ6YcffpDb75ggO3bsTOtQe/bukwl3TxQdbcsDAlEhsPPIMTlw/A/fp5Pq++W7QHaoNAQIbf117FwIwQsHcAAHcAAHcAAHcAAHcAAHcAAHcAAHcAAHcAAHCs+BQEPbsa9slR9/Pi46mtaMkNVRtjraVqdK1imT9T63OhrXrA/yNdsEI1WopCNsF735tkydNkN+P1gyVWyiYx4/flyefvY5efHlV9IemZuoLJZDIAoEUn2/onCOnENuCBDaFt7FBReEtBkO4AAO4AAO4AAO4AAO4AAO4AAO4AAO4AAO4AAO+HMgsNC2/fQNzn1qez3ydblAVu9ra4e2et/bIMNaU1a2cUE6oZLe2/bp52bLxMlTREfeuqdK1imRf9y+Xe67f7oMGDREXnzpFdF9eECgshNI5/tV2Rlx/t4ECG39dexcCMELB3AAB3AAB3AAB3AAB3AAB3AAB3AAB3AAB3AABwrPgUBC25rji+XrXUdk2uLtCcNYHWF7+MRp2fTTUekwY0PC7UwAm8mr96/701+abqh06tQp5161g6+/Ue6eNFlefe11ZwTua2+86YS1ulzvZavh7bgJdxPcpt8EbJlnAn+ePSsf7f3Zeer7IB/pfr+CPCZlRYMAoW3hXVxwQUib4QAO4AAO4AAO4AAO4AAO4AAO4AAO4AAO4AAO4IA/B7IKbYfN3SL7Dp6UIyf+lHXbDuUkiPUT3mYbT/gNlQ4ePCSffb5KHn3sCWfkrb5++NHHosvNY/eePQS3BgavoScwbu038j+ee8N56vsgH36/X0Eem7IKmwChrb+OnQsheOEADuAADuAADuAADuAADuAADuAADuAADuAADuBA4TmQVWirge2AJzc5z/0HT1a60DbdGITgNl1SbJdvAgM+XhcLbfV9kA9C2yBpVq6yCG0L7+KCC0LaDAdwAAdwAAdwAAdwAAdwAAdwAAdwAAdwAAe8HVi3bp0sWrRIFi5cyDMkDLQ9tF3y7Wxgoa0GuH5GxeZi22xjjFyGSgS32bYO+1cEgfW//CbV3vjAeer7IB+5/H4FWU/KCh8BQlvvi7t8X0BwfNoFB3AAB3AAB3AAB3AAB3AAB3AAB3AAB3AAB/w5oMFgcXGxHDlyJO8BIW1X1nbaHtou+Q5uswptzfTIGtjq+1wEsX7KzDZqyHWotHffPlmw8DU5dux4tlVlfwgUHIFcf78KDggVTpsAoW3ZxQMXUrDAARzAARzAARzAARzAARzAARzAARzAARzAgcJ1QEd0EtiGs/20XbR98vn9yiq09ROoVsS2aScACTYkVEoAhsUQCIAA368AIFbSIghtw3kRk8+LF46NEziAAziAAziAAziAAziAAziAAziAAziAA4XogE6JXIj1rix1znf7ENpaIQihkgWDtxAImADfr4CBVqLiCG25AK8sF4WcJ67jAA7gAA7gAA7gAA7gAA7gAA7gAA7gQLQdyHcoiF/J/cp3+2QV2u785URgscGOX05kPb1ytpUhVMqWIPtDIDEBvl+J2bAmOQFC2+QXElxowQcHcAAHcAAHcAAHcAAHcAAHcAAHcAAHcAAHCsOBfIeCeJLck3y3T1ahbUVMeeznGMl/7Z96LaFSakZsAYFMCfD9ypQc+xHaJr+Q4EILPjiAAziAAziAAziAAziAAziAAziAAziAAzhQGA7kOxTEk+Se5Lt9CG2tPGTTpk1y5swZawlvIQCBIAjo90q/XzwgkAkBQtvkFxJBXmidOHFCjh8/LseOHZOjR4/GnvpZl+v6II9HWRXXtrCGNQ7gAA7gAA7gAA7gAA7gAA7gAA7gAA7k34F8h4I4kNyBfLcPoa2VIOzYsUN+/fVXawlvIQCBIAjo90q/XzwgkAkBQtvkFxJBXGhpGOsOau3Q1n6v2xHe5r5NgmhXyqCdcAAHcAAHcAAHcAAHcAAHcAAHcAAHcCBcDuQ7FMSH5D7ku30Iba0EQUcR6WhADZgYcWuB4S0EMiSg3yP9Pun3Sr9fPCCQCQFC2+QXEtleaOl30w5l032v+2V7bPbPbdvCF744gAM4gAM4gAM4gAM4gAM4gAM4gAM4EC4HggoF33rrLdm8eTO/nzsZbPsG1T6Zfu8IbV0Jgv4SWkcEasikQQFPGOBA5g7o90i/TwS2rv9o+OiLAKFtsBce9gVDpoGtCXYJbnPXNnY78R7OOIADOIADOIADOIADOIADOIADOIADOBANB4IKBefPny8bNmwgtCW0XS3V7wjn01cKwMYQgAAEIFAQBMIW2n4xq7EUNSh7jnhrX9nF0fpZUtRglnwR8MVCLi7KdYpjE75m88pUydH4gSEXjlEmbuAADuAADuAADuAADuAADuAADuAADuBAvAOFGtrufn+GjHl+Y9nvQV2//0y1vqI9yLQ+QbVPpufLSNuCiCyoJAQgAIHKSyA8oe0+eePWxlJ061uyL3ZRUrLswfWlF185DG33vTVaimYlvjDyeyGQ7j1sUwW6Wo7fY7N9/MU6POCBAziAAziAAziAAziAAziAAziAAziAA5XDgaBCwaBG2jrh5pgZsmx3cv6pQtDE63fLshljZMwY+5n6eNl+HxLXJ/l5BtU+mdaf0Lby5iCcOQQgAIGCIBCW0NYJTeMCW48OvkBC25SjbHe8IcNjo4lHycIdR5OOymW0rYcLsWCfdZlepLIf7uAADuAADuAADuAADuAADuAADuAADkTNgaBCwWBCWw1UZ8jc52fIjPd3Jx2YkSoETby+JLSdu9FyeeNcGTNmrmzM4e/PEtfHqofH8YNqn0y9JbQtiMiCSkIAAhCovATCEdqWjKiNmwrZo1M/aULb3W/JiFjo6Z4ueaM8GFun0yy71msZGg47Zen6GTJLR/ja+8TCY1dZaY7ETX4v259k4YjGMrO4NKh1AtyHZe3RxMEt97ZNfrGX6UUa+8EVB3AAB3AAB3AAB3AAB3AAB3AAB3AAB6LlQFChYCCh7e5lMmPGMtltXuN+37lR5sZGx5YEu/HTI6dab9rNI7Q9qfua0baukbjWFMwavM54f2PcSN248Pdk/L72OkLbENzntvJGGpw5BCAAgegSyCa01al7v/nmG0k2ElTX6TZJL4BLQ9jYNMhxFzDmAuSklIS29hTKpVMqx8JU/Txa3ohNN+Jeb5UR26ekfOdeuimW6TYp63jypCSfGnmdzGxgj67Vz8lDW6ZIthxI5AbLk3/H4AMfHMABHMABHMABHMABHMABHMABHMCBSuCAn9BWB0r8+uuv8ssvv5R7zp49Wz777LNyy3Xb3377Tf7444+UPpWEojrCVsNPE6Lq77lKw9BYgOr3s/27spJ9ywWqpSNtNz5vHVfD41iYe1Kc4NX9WUNmx5OScstGCGsQXDZ6l9CW0Da6iQlnBgEIQCCPBLIJbY8cOSJLliyR4uJiz+BWA9vVq1fL0qVLk1/EOKGtHbbaFx7WezPS1rrATDWtcrn1HmVooFw+tE1z9K9VFxNMp7pP7dHih0tG9j60TtY+1FiKHlqXdHpkLc+Uzavlgwd7+MAHB3AAB3AAB3AAB3AAB3AAB3AAB3AAByqvA35C2++//16mTZsmU6ZMKfccP368TJw4sdxy3fbpp5+WgwcPpvh9nYae8SFnLAB1wtOydeprXAiaan3c78RKA9/YqF29t2182WXfh/iAN+6YWqZ9XH0fC3BLfNr4/Bgx4XC5fePqlNg/P+1TVu/E5fndhumR8xhEcGgIQAACEEhNIJvQVv+ibNeuXbJ48eJywa0JbDXU3b07+T0bTvoaaRs/3XG5UNa5QHBNa2xPkZx2aKsXKmXTMKczwtZcJKQKbXe9McqajtkedZt4imRTNq/BXaTBEpY4gAM4gAM4gAM4gAM4gAM4gAM4gAM4EC0HggoFs54e2QlANUC1niYE1XvOmvelYWdcCJpqfVxAGh/Eevrs3OO2rB4Jg1c7tHXtY84j4b5xdUrsVFDt43meadSB0DZ1XsAWEIAABCCQRwLZhLbaOXoFt74CW6czLQlZ076nrdUBu0NbZ8RsA2saY3dI6/5cWlb5kbbWxUUsvE1jNHCq6ZH1HrYj3pBd5h62zqjb5MEt0yNbbWG1faYXZ+wHTxzAARzAARzAARzAARzAARzAARzAARyIpgNBhYLZhrY6KjU2stb5fZaGq6VTFTuBaPxo2HKhrWu0bNz6uN+PJQ9ttR5l98qN37Zcme7Q1hUs29+ZcvvG1SmxW0G1j10XP+8JbfMYRHBoCEAAAhBITSDb0FY7RQ1ut2/f7oy41emQ9ZnWCFurM3eHr56drUfgGref1zTL7n3cn0vrkDS0dbZJf7pkvR9GotG2zihb13TIOkXyzOLEo2y1PE8eFj/WJ74YhA1scAAHcAAHcAAHcAAHcAAHcAAHcAAHcKByOBBUKJhdaKv3f7XuJVv6O7yyIFfX26FuyeeycDXVerst44PYeM9d9SgdPZtwtKwd2pbedzcueN69u/R+t67pnH38jjKo9ok/T5tH8veEtqnzAraAAAQgAIE8EggitNVO0gS3b7/9tu/AtqSTLQlFi259S/bFOnpXUOoRuJYPba1RtrERstaUyh5l6PHjynGOr8e29jtZMho4nWmSdaRxotD2qI60bWCNrHV/NiNwrVctL9MLEfZLfqEGH/jgAA7gAA7gAA7gAA7gAA7gAA7gAA7gQHQcCCoUzCa0dUaheo1StUfYxk2fPFc26rrnN5b9DjDV+tjvT5OFtqXhqpmi+fmNkvS+tHGhrTpRUraZGnmM3qN3d6krrgA43e9QUO2T7vHc2xHa5jGI4NAQgAAEIJCaQFChrXaAZqrkvXv3ll1gxC4g0rv4M9MbFzVo7Nz3VadM3rd7X0l5HoGrO2x1Ppfu6wTATnBrha8eZZR03vZ9cGfJF7s3yhdvzbLuPdtYUk7fbJ2rTmmcMLh1pkQuOT89z2SjbJkaOT1v3BdgfIYbDuAADuAADuAADuAADuAADuAADuAADlQ+B4IKBbMJbfEusXdBtU+mjAltU+cFbAEBCEAAAnkkEGRoq52lBreZdppR2i/paFtrFG3CYLd0G0bZJr7Ii5IvnAvtjAM4gAM4gAM4gAM4gAM4gAM4gAM4gAPZOxBUKEhom31bePkcVPt4lZ3Osv/K4+/hAz20/lKfBwQgAAEIRI9A0KFtOp1jZdkm2b1tU4W1up572ebm4rCy+Md54g8O4AAO4AAO4AAO4AAO4AAO4AAO4EBlcyCoUHDnzp3y22+/MTjFmlkwCJeCap9M60JoG718gzOCAAQgECkChLa5vXjPNLglsM1tu2R6Ycd+tAsO4AAO4AAO4AAO4AAO4AAO4AAO4AAOhNeBfIeCuJHcjXy3D6FtpKINTgYCEIBA9AgQ2ia/kAjiQkunOE56j1trumTdjimRc98mQbQrZdBOOIADOIADOIADOIADOIADOIADOIADOBAuB/IdCuJDch/y3T6EttHLNzgjCEAAApEiQGib/EIiyAstDWN1BK07wNXPupywtuLaIsh2pSzaDQdwAAdwAAdwAAdwAAdwAAdwAAdwAAfC4UC+Q0E8SO5BvtuH0DZS0QYnAwEIQCB6BAhtk19IcKEFHxzAARzAARzAARzAARzAARzAARzAARzAARwoDAfyHQriSXJP8t0+hLbRyzc4IwhAAAKRIkBom/xCggst+OAADuAADuAADuAADuAADuAADuAADuAADuBAYTiQ71AQT5J7ku/2IbSNVLTByUAAAhCIHgFC2+QXElxowQcHcAAHcAAHcAAHcAAHcAAHcAAHcAAHcAAHCsOBRYsWyZEjR4T2Cl97abto++SzbQhto5dvOGd05swZ+ezzVTJl6jSZOHmK76fup/trOWF5nD171qnT/dP/KXv37QtLtagHBCCQYwKEtuG7gMnnhQvHxgccwAEcwAEcwAEcwAEcwAEcwAEcwAEcwIFCdWDdunVSXFxMcHsyXA5rYKvtou2TT7cIbX2GDT9u3y5Dbx4hAwYNyfg586GH5dix4z6P7G/zr77+Rl56ZYH8fvCgHD5yxPdT99P9tZwwPDSw3bBhY4y9BtEEt2FoGeoAgdwTILQN1wVMPi9aODYu4AAO4AAO4AAO4AAO4AAO4AAO4AAO4AAOFLoDGgzqiE6dipdnOBhoe+Q7sFWvCW195g2nTp2Sb7/7Xjb9e3PGz127fhINIXP5+OTTT0Wf2TyCKCOb49v7Ku9RY8bKE089I+PvukeenTPXGT28f/8BezPeQwACESRAaMuFeKFfiFN/HMYBHMABHMABHMABHMABHMABHMABHMABHMABHEjlAKFtBAMOPSU7cN345ZdOwKmv5n2yKZN1G3cZFYlp+44d8uTTz8RGI+sUze+89558tmq1bPn2O5k0+T755ZdfnJHAGqDzgAAEok2A0JaLmVQXM6zHERzAARzAARzAARzAARzAARzAARzAARzAARzAgUJ3gNA2w6xDR8p++dVXsujNt9N6vvX2Etm3b3+GR/O/mx3anj592pkeWV/N+2RTJus2+rDL8F+DzPcwwazW0f1Its69LZ8hAIFoECC05WKr0C+2qD8O4wAO4AAO4AAO4AAO4AAO4AAO4AAO4AAO4AAOpHKA0DbDTENHf36w8kNnBGuyUatm3f3T/yk//PBDhkfzv1sQgWsQZfivucRG0xLaZkKPfSAQPQKEtlzMpLqYYT2O4AAO4AAO4AAO4AAO4AAO4AAO4AAO4AAO4AAOFLoDhLbRyzecM7IDVzMlcqFMj5xsNG2ydRFtSk4LApWeAKEtF1uFfrFF/XEYB3AAB3AAB3AAB3AAB3AAB3AAB3AAB3AAB3AglQOEtgHEITrq9ujRo84UxO5ph0+cOBHAEfwXYYe2Zkpkpkf2z5E9IACB/BMgtOViJtXFDOtxBAdwAAdwAAdwAAdwAAdwAAdwAAdwAAdwAAdwoNAdILQNII/YsWOn3DxilAwYNKTc88FHZsnx48cDOIq/IuzQ1t+eZVsHUUZZaem/SzaaNtm69I/AlhCAQCERILTlYqvQL7aoPw7jAA7gAA7gAA7gAA7gAA7gAA7gAA7gAA7gAA6kcoDQtpCSCx91tQNXpkf2AY5NIQCB0BEIS2j7008/yaZNm0TrwxMGOIADOIADOPC10y9q/5jqh85019PX8r3ie4UDOIADOBDvgP4MSl8bzwRH4IEDOIADOBCkA0H3ten+/Jtou4xC2507d8rzzz8vTz/9dKBPLVPLzuShjcSjjIAd2hbK9Mh79ux1ppg2o2kPHT4s+/bvl98PHoydmFmn01DzgAAEKgcB/f89USdWUcv1h2R96v+nPCAAAQgvNpeCAAAgAElEQVRAAAIQKCGg/aLpI7Ptk0059LXYBQEIQAACECgjQF9bxoJ3EIAABCAAgVwQCLKvzfbnYt0/o9BWw9Wff/45js/69etFn9k8tEwtO5NHGELbNWvXycTJU5znPx98SPbvP5DJqQSyz+erVsv7yz+Qs2fPZlSe7qf7azkV8dD7Ar/19mKZMnWafPyvT+See6fIRx99LKPGjJWvvvkmVoVTp07Jnr37Mj6vWEG8gQAECoZAGEJb/YsrfolcMMpQUQhAAAIQqEAC2j9qP5ntD6f0tRXYaBwKAhCAAAQKigB9bUE1F5WFAAQgAIECJBBUX5vtz8UZh7Y6wtb9CCK01TK9ynYfy+tzGEJbHRG66d+bnee3330vGjDm63Ho0CF59LEn5K6Jk2JBsgmU03nV/R574iln5GtFnYPyWrDwtdh9gYfePEKK16wloK2oBuA4EAgpgTCEtmHoY0LaPFQLAhCAAAQg4Nw2INsfTulrEQkCEIAABCCQmEAQPxfT1ybmyxoIQAACEIBAEH1ttj8XE9pG3EMdvXr06FEneNXphP08dT/dv6IfJrgdfP2N8tnnqwhsK7oBOB4EQkggDB0mP9yGUAyqBAEIQAACoSEQRF9NXxua5qQiEIAABCAQQgL0tSFsFKoEAQhAAAKRIhBEX0toaynBD/kWjAJ/e+r0aTlw4ACBbYG3I9WHQFAEwtBh0scE1ZqUAwEIQAACUSQQRF9NXxtFMzgnCEAAAhAIigB9bVAkKQcCEIAABCDgTSCIvjavoa2ZDtm8Ll68WPRpPmf6WsjTI3s3NUshAAEIQCAbAmHoMPlFcjYtyL4QgAAEIBB1AkH01fS1UbeE84MABCAAgWwI0NdmQ499IQABCEAAAqkJBNHXEtpanPkh34LBWwhAAAIRIhCGDpM+JkJCcSoQgAAEIBA4gSD6avrawJuFAiEAAQhAIEIE6Gsj1JicCgQgAAEIhJJAEH1tXkNbN1Uzsta93O9nRtr6Jcb2EIAABKJNIAwdJr9IjrZjnB0EIAABCGRHIIi+mr42uzZgbwhAAAIQiDYB+tpoty9nBwEIQAAC+ScQRF9LaGu1Iz/kWzB4CwEIQCBCBMLQYdLHREgoTgUCEIAABAInEERfTV8beLNQIAQgAAEIRIgAfW2EGpNTgQAEIACBUBIIoq8ltLWalh/yLRi8hQAEIBAhAmHoMOljIiQUpwIBCEAAAoETCKKvpq8NvFkoEAIQgAAEIkSAvjZCjcmpQAACEIBAKAkE0dcS2lpNyw/5FgzeQgACEIgQgTB0mPQxERKKU4EABCAAgcAJBNFX09cG3iwUCAEIQAACESJAXxuhxuRUIAABCEAglASC6GsJba2m5Yd8CwZvIQABCESIQBg6TPqYCAnFqUAAAhCAQOAEguir6WsDbxYKhAAEIACBCBGgr41QY3IqEIAABCAQSgJB9LWEtlbT8kO+BYO3EIAABCJEIAwdJn1MhITiVCAAAQhAIHACQfTV9LWBNwsFQgACEIBAhAjQ10aoMTkVCEAAAhAIJYEg+lpCW6tp+SHfgsFbCEAAAhEiEIYOkz4mQkJxKhCAAAQgEDiBIPpq+trAm4UCIQABCEAgQgToayPUmJwKBCAAAQiEkkAQfS2hrdW0/JBvweAtBCAAgQgRCEOHSR8TIaE4FQhAAAIQCJxAEH01fW3gzUKBEIAABCAQIQL0tRFqTE4FAhCAAARCSSCIvpbQ1mpafsi3YPAWAhCAQIQIhKHDpI+JkFCcCgQgAAEIBE4giL6avjbwZqFACEAAAhCIEAH62gg1JqcCAQhAAAKhJBBEX5u30Pb555+Xn3/+OQ7s+vXrRZ/ZPLRMLTuTBz/kZ0KNfSAAAQiEn0AYOkz6mPB7Qg0hAAEIQCB/BILoq+lr89d+HBkCEIAABMJPgL42/G1EDSEAAQhAoLAJBNHX5i203blzp8ybN0+efvrpQJ9appadyYMf8jOhxj4QgAAEwk8gDB0mfUz4PaGGEIAABCCQPwJB9NX0tflrP44MAQhAAALhJ0BfG/42ooYQgAAEIFDYBILoa/MW2oYRPT/kh7FVqBMEIACB7AmEocOkj8m+HSkBAhCAAASiSyCIvpq+Nrp+cGYQgAAEIJA9Afra7BlSAgQgAAEIQCAZgSD6WkJbizA/5FsweAsBCEAgQgTC0GHSx0RIKE4FAhCAAAQCJxBEX01fG3izUCAEIAABCESIAH1thBqTU4EABCAAgVASCKKvJbS1mpYf8i0YvIUABCAQIQJh6DDpYyIkFKcCAQhAAAKBEwiir6avDbxZKBACEIAABCJEgL42Qo3JqUAAAhCAQCgJBNHXEtpaTcsP+RYM3kIAAhCIEIEwdJj0MRESilOBAAQgAIHACQTRV9PXBt4sFAgBCEAAAhEiQF8bocbkVCAAAQhAIJQEguhrCW2tpuWHfAsGbyEAAQhEiEAYOkz6mAgJxalAAAIQgEDgBILoq+lrA28WCoQABCAAgQgRoK+NUGNyKhCAAAQgEEoCQfS1hLZW0/JDvgWDtxCAAAQiRCAMHSZ9TISE4lQgAAEIQCBwAkH01fS1gTcLBUIAAhCAQIQI0NdGqDE5FQhAAAIQCCWBIPpaQlurafkh34LBWwhAAAIRIhCGDpM+JkJCcSoQgAAEIBA4gSD6avrawJuFAiEAAQhAIEIE6Gsj1JicCgQgAAEIhJJAEH0toa3VtPyQb8HgLQQgAIEIEQhDh0kfEyGhOJXACDz2xFMyaMgN8vvBg06Z7s/uA+n6uyZOkhMnTrhXpfU5VflpFZLGRtu375AuPXrJ4iVL09iaTSAAASUQRF9NX4tLhUTAT5+k/aT2l7pP2B5hrlvYWFEfCOSbAH1tvluA4xcigTD+bPfFho3Oz5tat6Af9OtBE6W8ykYgiL6W0Nayhh/yLRi8hQAEIBAhAmHoMOljIiQUpxIYAfsX1hrEaiCrYWeiHz51+3RDWw1M7W3TKT+oEwvjD/ZBnRvlQCBXBILoq+lrc9U6lOsmoP1RNgGq3z7J9Ct2v+auU1Cf9ViDhlyfsC92H4df7rqJ8BkC4SVAXxvetqFm4SVg+uAw/UGun9CWfj28blGzaBIIoq8ltLXcUKA8YYADOIAD0XQgiA4vmzLUKx4QgEA8ATu0jV/j/clPaOtnW++jZb40jD/YZ3427AmBiiGg/WQ2/azuS19bMW1V2Y9iAtdsQtswM/Tzi2A9D0LbMLcmdYNAPAH62ngefIJAOgTC+LOdn77az7bKg349HSvY5v9n7024qzjSbNH7W+5bb9273nqvV7/uLt9+dvt2d3Xf6uq5qvva3V1TV5XtqnK5XJ5nPAPGGDCTMbPBzGKSmBHzKAQChBDzDBoYBBoACSGh760deb48kXEyzyClpDzSDtYhM2P8Ykec3Irc54skAtEIxMG1fV0Xo/x/iTaxuFKSAmgcg8I6Ovv80IcYEkPOgaEzB5Jwf4cNDERgKCKAXxw/9viTgsWgBjzIVo9ZnCNdP/YvlJGWbXtkXTCiLPLBw8j2Moqq243HNYLbnj54V9s0H/Lq4nzrtu2mbbUBNuUKWtbuq90X1JUtTe1wy2h8rvaZTgSKEYE4uJpcW4wjX1w2u7yh93O9X+/bX2F4CvHgRdy3ca4f+96PNOVALb+0ZLlfXnkUCGm68oDyTDaOQvvarh5tDg1D3i2j9rn9Vjuy2aZl1Xa1wcUANqHfYelhNmq
Step 1: Imports and initial set-up
Start your Weights & Biases run. If you don't have an account you can sign up for one for free at www.wandb.ai
Step 2: Set up your API key
Set up your OpenAI API key for authentication.
Step 3: Dataset Preparation
Download a dataset from LegalBench, a project to curate tasks for evaluating legal reasoning, specifically the Contract NLI Explicit Identification task. This comprises a total of 117 examples, from which we will create our own train and test datasets.
Step 4: Format our Data for Chat Completion Models
Modify the base_prompt from the LegalBench task to make it a zero-shot prompt. Split it into training/validation dataset, training on 30 samples and testing on the remainder.
Step 5: Save the data to Weights & Biases
Save the data in train and test files first. Validate that training data is in the correct format using OpenAI fine-tuning documentation script. Log data to Weights & Biases Artifacts for storage and versioning.
Step 6: Create a fine-tuned model
Download training & validation files and save them to a folder called my_data. Retrieve the latest version of the artifact. Upload the training data to OpenAI for processing.
Step 7: Time to train the model
Define ChatGPT-3.5 fine-tuning hyper-parameters and start training. This takes around 5 minutes to train, and you get an email from OpenAI when finished.
Step 8: Log OpenAI fine-tune jobs to Weights & Biases
Call openai wandb sync to log all un-synced fine-tuned jobs to W&B. Pass your OpenAI key as an environment variable, the id of the fine-tune job, and the W&B project where to log it.
Discussion
Questions & comments · 0
Sign In Sign in to leave a comment.