FAQ
General Questions
Q1: What is LangTest, and what does it do?
A: LangTest is a tool that helps developers test their custom language models. It generates questions, checks the model's answers, and provides a detailed report on how well the model performs.
Q2: What are the system requirements to use LangTest?
A: LangTest is a web-based application accessible via modern browsers. Ensure a stable internet connection and sufficient credits for performing tests.
Q3: Does LangTest support multiple languages?
A: Currently, LangTest primarily supports English language testing. For support with other languages, please reach out to our support team.
Q4: Does LangTest currently support image-based conversation model testing?
A: No, image-based testing is not currently supported in LangTest. However, we're continuously working on expanding our capabilities. For the latest updates and information, please contact our support team.
Importing Models
Q4: How do I import a custom model into LangTest?
A: Go to the AgentSellar, select Import Model, and enter details such as Model Name, Model URL, and configure system instructions like authentication parameters, custom prompts, and request/response mappings. Complete the import by verifying the model.
Q5: Can I import a model that requires authentication?
A: Yes, during the import process, you can add authentication tokens or keys under Authentication Parameters to ensure secure access to your model.
Q6: What types of system instruction can I configure when importing a model?
A: You can configure additional settings like custom prompts, request payload mapping, response mapping, and other key parameters to ensure compatibility with LangTest’s features.
Question Generation and Testing
Q7: How does LangTest generate questions from content?
A: In Paste & Ask and Upload & Ask, LangTest uses pasted or uploaded content to generate questions based on the selected type (single choice, multiple choice, descriptive) and difficulty level (easy, medium, hard).
Q8: Can I configure question types and difficulty levels in LangTest?
A: Yes, LangTest allows you to select single choice, multiple choice, or descriptive questions and adjust difficulty to easy, medium, or hard for more targeted testing.
Q9: How does LangTest compare my model’s response with a generic model?
A: LangTest compares your model's answers to answers from a generic model. It checks if your model's answers are correct or incorrect. For free texts, it also calculates a similarity score to show how close your model's answer is to the expected answer.
Q10: What is the purpose of similarity scoring in LangTest?
A: Similarity scoring helps us understand how well your model's answers match the expected answers. It's like comparing your model's work to a generic model's work. A higher score means your model is doing a better job. This is only for questions with text-based answers.
Reports and Analysis
Q11: What information is included in LangTest reports?
A: Reports provide summaries of generated questions, model responses, similarity scores, and overall testing results. They offer insights into model performance and help identify areas for improvement.
Q12: How do I access and interpret similarity scores in LangTest reports?
A: The similarity score is a measure of how closely the model's answer matches the expected answer for descriptive questions. You can find this score in the LangTest reports. It's calculated for questions where you've provided an expected answer. Higher similarity scores indicate greater accuracy.
Q13: Can I download LangTest reports?
A: Currently, LangTest provides in-app access to reports. For export options, check for updates or contact support for additional assistance.
Tracking Credits and Usage
Q14: Where can I check my credit balance in LangTest?
A: Go to the Billing section, where you can view your remaining credits, as well as usage details in the activity report, daily usage report, and credit ledger.
Q15: How are credits consumed during testing?
A: Credits are deducted for each question and answer generated during the trial period. The specific credit usage may differ depending on the plan you choose.
Q16: What happens if I run out of credits?
A: If you run out of credits, testing actions will be temporarily suspended until credits are replenished. You can add more credits through the Billing section or contact support.
Troubleshooting and Support
Q17: Why did my model import verification fail?
A: Model import failed. Please review your model's API endpoint, authentication credentials, and request/response mappings to ensure they are correct and accessible. Double-check your configuration and make necessary edits.
Q18: What should I do if LangTest is not generating questions correctly?
A: Ensure that the pasted or uploaded content is clear and formatted properly. Also, verify question settings for type and difficulty. If issues persist, contact support for troubleshooting.
Q19: Why am I seeing low similarity scores?
A: Low similarity scores may indicate that the model’s responses are not closely aligned with expected answers. Consider refining the model’s configuration or testing with different prompts to improve accuracy.
Q20: How do I contact LangTest support for assistance?
A: For technical issues or additional assistance, please contact support@synergetics.ai or visit our help center.
Advanced Features and Future Updates
Q21: Can LangTest support multiple models in a single session?
A: Currently, each model is tested individually, but multiple models can be imported and tested separately within LangTest.
Q22: Are there plans to add new question types or testing configurations?
A: LangTest is continually evolving, with potential updates based on user feedback. Keep an eye on the Updates section or contact support for information on upcoming features.
Q23: How can I stay informed about LangTest updates?
A: Subscribe to Synergetics newsletter or check the updates section on the website for the latest news and feature enhancements.