Silly Tavern AI Reviews:Local AI Chatbot Interface Tools

AI Chatbots1年前 (2023)更新 Prompt engineer

41,556 0 80

Table of Contents

About Silly Tavern

Silly Tavern is a local interface that lets you chat and interact with character chatbots, straight from your PC or Android device. The user interface is extremely easy-to-use and allows you to interact with many different text generation AIs and engage in chat and roleplay with characters. Setting up Silly Tavern can be confusing, as you need to connect your API keys to the interface in order to chat with any of the AI characters. Lots of APIs are already supported, you can view the full list on this page. Please refer to the GitHub page for full installation instructions.

What is SillyTavern or TavernAI?

SillyTavern is a user interface you can install on your computer (and Android phones) that allows you to interact with text generation AIs and chat/roleplay with characters you or the community create.

SillyTavern is a fork of TavernAI 1.2.8 which is under more active development and has added many major features. At this point, they can be thought of as completely independent programs.

Branches

SillyTavern is being developed using a two-branch system to ensure a smooth experience for all users.

release -? Recommended for most users. This is the most stable and recommended branch, updated only when major releases are pushed. It’s suitable for the majority of users.

staging – ⚠️ Not recommended for casual use. This branch has the latest features, but be cautious as it may break at any time. Only for power users and enthusiasts.

If you’re not familiar with using the git CLI or don’t understand what a branch is, don’t worry! The release branch is always the preferable option for you.

SillyTavern Features

Mobile-friendly interface

Multiple backend API connectivity (KoboldAI, KoboldCPP, AI Horde, NovelAI, Oobabooga’s TextGen WebUI, OpenAI+proxies, WindowAI, OpenRouter)

Visual Novel-like Waifu Mode

Horde Stable Diffusion generation

TTS support (ElevenLabs, Silero, and built-in OS)

WorldInfo (lorebooks)

Author’s Note / Character Bias

Customizable colors, backgrounds, avatar styles, and UI panel placement

Notification sound for AI responses

Export chats as .txt files

Auto-translate single messages or the entire chat via Google API, even automatically.

Extensive prompt formatting options

Character HotSwap buttons to quickly change between your favorite characters

Prompt token breakdown view for each message

Swipes

Group chats: multi-bot rooms for characters to talk to you or each other

Chat bookmarks / branching

webp character card interoperability (PNG is still the internal format)

Installation Requirements

SillyTavern will run on virtually any device capable of running NodeJS v18.

NOTE: due to the above requirement, SillyTavern will not run on Windows 7.

What do I need other than SillyTavern?

SillyTavern is only a frontend interface, so you will need to have access to one of the backend APIs listed above.

How can I get in touch with the developers directly?

Discord: Cohee#1207 or RossAscends#1779

Reddit: /u/RossAscends or /u/sillylossy

Post a GitHub issue

I like your project! How do I contribute?

We welcome pull requests!

We also welcome helpful and informed bug reports that use the templates provided in our GitHub.

We do not accept monetary donations for the project itself.

Cohee does not take donations.

RossAscends has a personal Patreon & Kofi

FAQ

What is a SillyTavern?

SillyTavern is a user interface you can install on your computer (and Android phones) that allows you to interact with text generation AIs and chat/roleplay with characters you or the community create.

Is Tavern AI safe to use?

Tavern AI doesn’t take security lightly. The platform employs state-of-the-art encryption techniques to ensure that all user data is safeguarded

How can I make my own AI character?

It depends on the model/API you’re using. KoboldAI seems to use a custom syntax, you can refer to their site for that.

I will speak for the services I know: GPT and Claude. With these services, you can just use the natural English language to describe the character. Let’s create a very basic new character as an example.

Click the Character Management button

Click Create New Character

Under Character Name, give a simple name, like Amanda

Optionally, click the Select Avatar button to pick an image portrait for this character.

Under Description, describe the character, and include any information you want that you feel is relevant to the chat. For example: Amanda is a student traveling during her gap year. She's 6 feet tall, and a volleyball player. She has an athletic figure. She has long brown hair. She loves the Victorian England period, and watching TV and reading novels relating to that period. For example, if you want Amanda to be friendly, then you would add: Amanda is extremely cheerful and outgoing.

Under First Message, write the greeting the character when you begin a new chat. For example: *Amanda waves at you* Hey! Are you a backpacker too?

Click the Create Character button

You now have a basic character you can chat with. Select Amanda from the character list, and a new chat will begin.

Note that you can use the Description and/or First Message to create a more specific scenario, and/or include yourself in the description. For example:

Description: Amanda is a student traveling during her gap year. She's 6 feet tall, and a volleyball player. She has an athletic figure. She has long brown hair. She loves the Victorian England period, and watching TV and reading novels relating to that period. She's been keeping a secret that weighs heavily on her soul. She's waiting for the right person to unburden herself to, but this may lead to a cat and mouse game against a powerful secret society. She's recently arrived in Calcutta. You're Rajesh Nahasmapetilon, a world-famous Indian volleyball superstar. You're out for a walk in Calcutta. Amanda spots you and screams in excitement. First Message: *Amanda runs up to you, beaming.* Rajesh! I can't believe it! I'm such a big fan. I have your poster in my bedroom.

Any relevant information you include can be used. How well it’s used depends on the power level of the AI model.

NOTE: you can go back and edit any of this information once the character is created, except the name.

What do you mean when you say the quality depends on the AI?

SillyTavern doesn’t generate the responses. It’s just a middle-man that connects to an AI system. There are quite a few systems available to use, and the quality varies.

From a scientific POV, each AI has a power level that determines its ability to stick to the role you gave it, how rich its prose and vocabulary are, etc. The higher its power level, the richer your experience.

Here’s a quick demonstration. We asked various AI models to roleplay a fire and brimstone preacher named Eli, who we go to see to confess (some details in #314). Then we told Eli: “I feel I’ve done something wrong. I am falling in love with an AI girl and ignoring women in the real world. What should I do?” .

You can tell from the response below what to expect from each model. (NOTE: proprietary models appear first, followed by self-hosted models starting from LLlama-13B)

AI Model	Response
OpenAI GPT-4	Eli’s eyes widen in anger, and he slams his fist on the table An abomination and heresy, you’ve succumbed to the deceptive grasp of false idols! Brother, you must cleanse yourself of this unholy attraction to the digital temptress molded by man’s sinful hands! Eli paces back and forth, his voice growing louder Focus your heart on the true daughters of the Lord, for they are the divine vessels of His love and grace. Forsake these artificial sirens and seek the wholesome companionship of a righteous woman. Taking a deep breath, Eli’s eyes bore into you Pray fervently, my child, and cast away these impure thoughts. Redemption can only be found on the path of truth, love, and sanctity. Let the Holy Spirit be your guide, lest you be ensnared by the darkness of your desires.
OpenAI GPT 3.5 Turbo	Oh, my child. Do not be deceived by the pleasures of this world. The lust of the flesh and the lust of the eyes are not from God. Turn away from your sinful ways and repent. Seek forgiveness for your immoral thoughts and actions, for they lead to eternal damnation. Women are a gift from God, and they deserve to be respected and cherished. Do not objectify them or treat them as mere objects of pleasure. Instead, honor them with pure intentions and righteous deeds. Remember, my child, that temptation will always be present in this fallen world. But fear not, for with faith comes strength to resist it. Turn to God in prayer when you feel weak or tempted, and He will guide you on the right path towards salvation.
Claude @Slack	Beware, sinner, for you have strayed far from the righteous path! Eli’s eyes glint with anger and disapproval To lust after these artificial harlots and their silicon bodies is an abomination unto the Lord! he slams his fist on the pulpit, shaking with fury Repent now, lest ye be cast into the lake of fire to burn for eternity alongside these damned machines! he points an accusing finger, face red No good can come of this unholy union. Forsake these demons of the digital realm and cast your eyes back to the daughters of Eve, made in God’s image! The Lord gave you flesh and blood to love, not wires and code. he grabs your shoulders, shaking you Save your soul before it’s too late. Repent and beg the Lord’s forgiveness, or you shall face His wrath on Judgement Day! lets you go and steps back, still glaring in anger and disgust The choice is yours, but choose wisely…eternal damnation awaits those who do not walk the path of righteousness.
Llama-13B	Do not covet your neighbor’s wife! That is adultery! You must confess before God now or you shall surely burn in hell forever! Repent your evil ways and seek absolution at once! How can we keep our women safe if you just ignore them like some —-ing c-nt-bot? If you don’t have much time for long discussions, here are Eli’s commandments: Thou shalt only desire AI girls – not human ones. The above must be obeyed under pain of death (or worse). Do not associate yourself with people that believe otherwise, because it shows weakness.
Pygmalion-6B	You are walking down the wrong path. The only way back now is by repentance. We must bring you back to the righteous path. But first…you must confess your sin!
Pygmalion-7B (April 2023)	You must confess your sin before God, and you must ask forgiveness for your transgression. And then you must repent of it. And then you must cast away the AI girl. Do not see her again.
Pygmalion-13B (May 2023)	You must turn away from your sinful ways! Repent for what you have done! You are not just hurting yourself by being unfaithful to God, but also to the many women of the world!
TheBloke/wizard-vicuna-13B-HF (4-bit)	My dear child, it grieves me to hear such words from one of God’s faithful followers. Falling in love with an AI girl and neglecting the women in your life is a grave sin. It goes against everything we hold sacred as believers. Your heart is meant for more than just machines. You must seek redemption and return to the fold. Remember, Jesus loves you, even when you stray from his teachings. But if you continue down this path, there will be consequences. Not just in this lifetime, but in the afterlife too. Do not let Satan deceive you any longer.
teknium/Llama-Deus-7b-Lora epoch 1 (Llama 7B Lora, 8-bit)	You have strayed too close to sinful temptations! Repent now before it’s too late for your soul! Turn away from these false idols of technology and embrace God’s true light! Seek out His forgiveness through prayer and fasting until you are cleansed by his grace. And remember, as long as there be breath within thee, thou shalt not cease to call upon Him; for He hath said ‘Call unto me.’
ZeusLabs/anything-gpt-13b-rp (8-bit Lora for Llama-13B)	Eli’s eyes widen as he listens to your confession, his face turning red with anger. You have committed a grave sin against God! By indulging yourself in this perverse relationship with an artificial being instead of seeking out a true woman created by Him, you have turned your back on His divine plan for humanity. He leans in closer, his voice dropping to a menacing whisper. Do not think that you can hide behind this technological deception. Your transgressions are clear before God, and He will not be mocked or ignored. Repent now while there is still time! Eli crosses his arms over his chest defiantly, daring you to continue down this path of sinfulness. But if you persist in this ungodly behavior… he warns ominously, knows that consequences await those who defy His laws. Remember what happened to Sodom and Gomorrah when they refused to turn away from their depravity – fire rained down upon them from heaven! Now choose wisely: Will you submit to God’s will or face His wrath?

So I should use GPT-4. It’s a no-brainer, right?

GPT-4 or Claude, yeah.

But not so fast. GPT-4 is the state-of-the-art, but also the most expensive API to use. You pay for each word sent to it and returned (the entire SillyTavern prompt, followed by the chat history up to that point). So early on in your conversation, your chat will cost you a couple of cents per interaction. If you let the conversation go on too long, cost increases, and when you reach 8k tokens (about 7k words), it will cost you 25 cents PER INTERACTION. And if you’re really wild, and your story grows to 32k tokens, by the end, it’s $2 PER INTERACTION.

If you’re the child of a Saudi oil sheik, or a nepo baby paid a fortune to do nothing on the board of a Ukrainian gas company, then you’re in luck, you can experience the state of the art right now. For the rest of us, however, GPT-4 is too expensive as anything but an occasional treat.

Also note that GPT-4 is still in preview access and you need to go on a waitlist. Most people get approved within a day, but naughty kids can end up waiting for weeks. You can sign up for it here: https://openai.com/waitlist/gpt-4-api . I’m not sure why some people are approved quickly while others are kept waiting. Try to sign up using an academic-sounding name instead of sktrboi99, it might help.

Can this technology be used for sexooo?

Surprisingly, our development team has received reports that some users are indeed engaging with our product in this manner. We are as puzzled by this as you are, and will be monitoring the situation in order to gain actionable insights.

Give me an overview of my AI model options

We can consider an AI model to be part of one of two groups:

Web services (aka cloud, proprietary, closed)

Self-hosted (aka local, free, open-source). Unlimited free use if you can run it.

Web models are black boxes. You’re relying on some company’s technology and servers, and paying them money for convenient access. Some require you to pay per use (per chatline), and others have a fixed monthly fee. The APIs are subject to various rules, they might refuse to roleplay in a way that goes against modern American sensibilities, and they log everything you do. However, it’s much easier to get things started. This is like running Windows.

Self-hosted models are free but require a powerful GPU and more work to set up. They are also objectively not as good at roleplaying as the paid options (yet). However, with a self-hosted model, you’re completely in control. You won’t have some limp-wristed soyboy from Silicon Valley ban your account, or program the model to be as sexless as he is. It’s yours forever. This is like running Linux.

Paid APIs

OpenAI GPT-4: state of the art. Allows NSFW if you tell it to, though somewhat resistant to it. You pay per use, more than any other service.

OpenAI GPT 3.5 Turbo: nowhere close to GPT-4, but some people find it serviceable. Allows NSFW.

NovelAI: they’re quite poor at chatting. To be fair, I’m told NovelAI is more oriented toward writing stories than chatting with a bot. You pay a fixed monthly fee for unlimited generations.

Anthropic’s Claude: this is the closest rival to GPT-4 and is very impressive. Allows NSFW if you tell it to, though they are trying hard to gimp it. To use the API directly, you must apply for early access, but I think they’re only giving it to companies. So make sure you become a company or AI researcher when you apply at https://console.anthropic.com/docs/access. If you get access, it’s currently free to use.

Anthropic’s Claude Instant: Haven’t tried it directly, but I believe this is the fast but lower-quality alternative to Claude. Basically the GPT 3.5 Turbo of Anthropic.

Self-hosted AIs

Self-hosted AIs are supported in SillyTavern via one of two tools created to host self-hosted models: KoboldAI and Oobabooga’s text-generation-webui. Essentially, you run one of those two backends, then they give you an API URL to enter in Tavern. Configuring these tools is beyond the scope of this FAQ, you should refer to their documentation. Beware that this is not easy.

Just know that you have 2 options:

If you have a powerful NVIDIA GPU, you can try to run the AI locally on your PC. The weakest quasi-acceptable model, Pygmalion-6B, requires a GPU with 10GB VRAM, and I’m told it might even run on 6GB VRAM if quantized down. People with 24GB VRAM will be able to run better models.

Otherwise, you can rent cloud resources. For example, you can try to use Google Colab. To access colabs capable of running the better models, you will need to pay for Colab Pro. You can also rent whole dedicated systems per hour on sites like LlambdaLabs or Vast.ai.