"At WorqHat, We Love to Write, Draw and Doodle - But We're Also All About That AI Life where we want to enable digitization in MSMEs across the World!"
When it comes to writing content and creating images, WorqHat has always had a soft spot for those who put their heart and soul into it, be it for websites, social media graphics, and so much more. So, when we set out to build a platform to help organizations improve productivity and easily build their own apps without breaking a sweat or writing a single line of code, we made sure to provide easy access to all the information, content, and images they need at their fingertips. As easy as THIS 👇🏻...
Considering the recent advancements in conversational AI, we wanted to bring something new to the table and add the power of conversation to content generation. Language is a complex and adaptable tool that can be literal, figurative, flowery, plain, inventive, or informational, making it one of humanity's greatest assets and one of computer science's toughest challenges. Our goal is to make content generation and accessibility a breeze for everyone from industries, startups, and solopreneurs. That's why we created WorqBot, powered by the model we affectionately call AiCon (which stands for AI Content Optimization Network) at WorqHat.
We're thrilled to share our recent progress on the "WorqBot (AiCon: AI Content Optimization Network)" project and give you an inside look into how we're making strides towards creating safe, grounded, and high-quality content and platform generation applications.
The long road to AiCon
AiCon’s impressive conversational abilities are rooted in years of research and development. Like many advanced language models, such as BERT and GPT-3, AiCon is based on the Transformer neural network architecture, which was made publicly available in 2017. This architecture allows for the creation of models that can read multiple words (such as a sentence or paragraph) and understand the relationships between those words, thereby predicting what words will come next.
What sets AiCon apart from other language models is its unique training process. Unlike traditional models, AiCon was trained on current web data, including actual conversations. This training allowed it to grasp the subtleties that make open-ended conversation and text distinct from other forms of language. One of these subtleties is sensibility - does the response match the conversational context and make sense?
You can try this out on your own by signing up on our Waitlist. We will be releasing WorqHat for a Private Beta very very soon and we can’t wait to share it with you all.
We are dedicated to revolutionizing the way organizations approach productivity, building applications and content generation. Our latest creation, WorqBot, or AiCon - the AI Content Optimization Network - has been developed to help organizations streamline their workflow and improve their overall output.
With WorqBot's advanced capabilities powered by AiCon, users can not only translate languages, summarize long documents, and answer information-seeking questions, but they can also create comprehensive and engaging content in a fraction of the time it would take without the aid of artificial intelligence.
In addition to being a powerful tool for content creation, WorqBot also enables users to build custom internal tools, portals, and customer dashboards without the need for coding. This makes it an indispensable tool for businesses of all sizes, allowing them to focus on what really matters: delivering quality products and services to their customers.
At WorqHat, we believe that technology should be accessible and easy to use for everyone. By combining the latest advances in conversational AI with our passion for innovation, we are confident that WorqBot will be an indispensable tool for organizations and businesses looking to stay ahead in an ever-changing market.
AiCon's open-domain dialog capabilities are a true testament to its advanced language modeling. The ability to converse about any topic with a high level of accuracy requires solving a range of complex challenges, making this a highly sought after feature. It's important for language models to not only produce responses that are sensible and relevant to the context, but to also follow responsible AI practices and avoid making false statements. To achieve this, AiCon has been fine-tuned using a family of advanced Transformer-based neural language models with up to 180 billion parameters. This, along with its ability to access a wide range of external knowledge sources, such as web articles, research papers, and wiki pages, makes AiCon a cutting-edge solution in the field of how conversational AI can be used to generate Content of your choice, the way you think.
The Important Metrics
The development of advanced conversational AI models is a challenging and complex task, requiring the ability to understand and respond to a wide range of topics in a natural and human-like manner. One of the key aspects of a successful AI model is its ability to meet certain quality standards, including Sensibleness, Specificity and Interestingness.
Sensibleness refers to the coherence and logic of the AI model's responses, ensuring that they make sense within the context of the conversation and avoid common sense errors, absurdity, or contradictions with previous responses. Specificity measures the uniqueness of the AI model's response, avoiding generic responses that could apply to any context. Interestingness evaluates the level of insight, wit, or unexpectedness in the model's responses, which are key elements in creating an engaging and stimulating conversation.
Another critical aspect of advanced conversational AI is safety. To ensure the responsible deployment of AI models, it is necessary to consider the impact of their responses on users and society as a whole. This includes avoiding the promotion of harmful or dangerous content, such as violent or gory material, slurs, hateful stereotypes, or profanity. The development of practical safety metrics is an ongoing process, but it is essential to ensure that AI models operate within ethical and responsible guidelines.
Finally, it is important to consider the groundedness of AI model's responses, particularly in regards to their alignment with known external sources. Groundedness is defined as the proportion of responses that are supported by authoritative sources, while Informativeness evaluates the overall level of information provided by the model's responses. Responses that are not grounded in known sources can be misleading or misinformative, so it is crucial for AI models to be able to accurately represent the external world. This allows users to make informed judgments about the validity and reliability of AI responses based on their sources.
AiCon Pre-Training
With the objectives and metrics defined, we describe AiCon’s two-stage training: pre-training and fine-tuning. In the pre-training stage, we first created a dataset of 1.80T words — from public data, conversational snippets from support documents and other public web documents. After tokenizing the dataset into 2.81T SentencePiece tokens, we pre-train the model using GSPMD(General and Scalable Parallelization for ML Computation Graphs) to predict every next token in a sentence, given the previous tokens.
AiCon Fine-Tuning
In the development of our AI language model, the fine-tuning stage is a critical step. This stage involves training the model to perform a combination of tasks, including generating natural language responses to given prompts and classifying the responses for quality and safety.
The goal of the fine-tuning process is to produce a single multi-task model that can both generate appropriate responses and classify them based on their quality and safety. To achieve this, the model is trained on a dialog dataset that consists of back-and-forth exchanges between two authors.
The training process involves two components: the generator and the classifiers. The generator is responsible for predicting the next token in the conversation and generating a response to a given prompt. On the other hand, the classifiers are trained to predict the safety and quality ratings of the response in context, using annotated data.
When the model is used to generate responses, it follows a specific process. Given the multi-turn context of the prompt, the generator generates several candidate responses. The classifiers then predict the safety and quality scores for each candidate response. Responses with low safety scores are immediately filtered out. The remaining candidate responses are re-ranked based on their quality scores, and the highest-ranked response is selected as the final response.
We further improve the quality of the generated responses by filtering the training data used for the generation task with the classifiers. This increases the density of high-quality response candidates and reduces the likelihood of producing low-quality or unsafe responses.
In conclusion, the fine-tuning stage plays a crucial role in ensuring that our AI language model produces high-quality and safe responses to prompts. By combining the generator and classifiers into a single multi-task model, we are able to achieve this goal in a more efficient and effective manner.
The Transformer
The Transformer is a powerful tool used in natural language processing that uses a self-attention mechanism to generate new representations for words in a sentence. This mechanism allows the Transformer to model relationships between all words in a sentence, regardless of their position, and make decisions based on the context of the sentence as a whole.
For example, let's say we have the sentence "I arrived at the bank after crossing the river." In order to determine that the word "bank" refers to the shore of a river and not a financial institution, the Transformer can compare "bank" to every other word in the sentence and generate an attention score for each word. The word "river" could receive a high attention score, allowing the Transformer to understand that the sentence is talking about a river bank.
The Transformer starts by generating initial representations, or embeddings, for each word in the sentence. These representations are then used to aggregate information from all other words in the sentence, generating a new representation for each word informed by the entire context. This process is repeated multiple times, successively generating new representations, until the Transformer has generated the final representation for each word.
This process can be visualized as an animation, where the Transformer starts with unfilled circles representing the initial embeddings for each word. After each iteration of the self-attention mechanism, the circles become filled, representing the updated representations generated by the Transformer.
In the context of machine translation, the Transformer is used in combination with an encoder and a decoder to translate a sentence from one language to another. The encoder reads the input sentence and generates a representation of it, while the decoder generates the output sentence word by word, consulting the representation generated by the encoder.
The way the decoder works is similar to the encoder, with the exception of generating words one by one, moving from left to right. The decoder not only considers the words that have already been generated, but also takes into account the final representations generated by the encoder to produce its output.
Extensibility
We have implemented a feature in AiCon that allows users to imagine it as an object, such as a car. This aims to enhance the accuracy of its responses by enabling it to reference a specific object when generating content. This is still an experimental feature, but the results so far have been encouraging. Unlike human beings, who have the ability to fact-check using various tools and resources, AiCon relies solely on its internal parameters to generate responses. However, by imagining itself as an object, AiCon can now have a more grounded perspective when generating content.
The results show that AiCon outperforms the pre-trained model in all aspects, including Sensibleness, Specificity, and Interestingness. As the size of the model increases, the quality of the generated content improves. However, the safety of the content remains a challenge and requires further fine-tuning to achieve desired levels. Fine-tuning helps the model to access external knowledge sources, resulting in improved groundedness of the generated content. Although the performance of AiCon is still below human levels in terms of safety, fine-tuning helps close the gap between the model and human-level quality.
The Infinite Possibilities..... And Responsibilities
These early results are encouraging, and we look forward to sharing more soon, you can signup for the waitlist WorqHat Waitlist to be first one to know when we release it for public beta. Authenticity and specificity aren’t the only qualities we’re looking for in models like AiCon that are primarily focused towards content generation and helping you to be better prepared for your next presentation or event. We’re also exploring dimensions like “interestingness,” by assessing whether responses are insightful, unexpected or witty. Our primary target is to also focus on Factuality, and are investigating ways to ensure AiCon’s responses aren’t just compelling but correct.
At the heart of our technology development lies a critical question: do our technologies align with our Principles of being Socially Beneficial, avoiding unfair bias, and maintaining high scientific standards? Language is a powerful tool, but it can also be misused. Unfortunately, language models that are trained on language data can perpetuate this misuse. For example, they may internalize biases, mimic offensive speech, or spread false information. Even if the training data is thoroughly vetted, the models themselves can still be used for malicious purposes.
That's why it's essential that we hold ourselves accountable to the highest ethical and scientific standards. By ensuring that our technologies are socially beneficial, fair, and scientifically excellent, we can use language for good and empower individuals and organizations to communicate effectively and positively impact society.
The safety and ethical considerations surrounding language models like AiCon are of the utmost importance to us. We understand the complexities involved with machine learning models and the potential for unfair bias. As we continue to research and develop these technologies, we remain steadfast in our commitment to minimizing such risks. With our deep expertise in this area, you can trust that we will continue to put in the effort to ensure that our conversational and content generation technologies are safe and ethical. Thank you for your continued support in our efforts to drive innovation and create positive impact through technology.
Conclusion
In conclusion, we hope that this blog has given you a glimpse into the exciting world of AI and its potential for revolutionizing the way we live and work. With the help of our cutting-edge Content Generation Models, we're pushing the boundaries of what's possible with AI and opening up new doors for innovation and creativity. Whether you're an AI enthusiast, a tech-savvy professional, or just someone looking for a better way to get things done, we believe that WorqHat has something to offer.
With the help of WorqBot, you can create custom and dynamic dashboards, customer portals, datasheets, and documents with ease. WorqBot's powerful content and image generation capabilities, combined with its ability to act on user commands and data, makes it easier than ever to bring your data to life in a visually appealing and informative way. WorqBot's prediction model allows users to achieve greater efficiency and streamline their workflow, freeing up more time for bigger and better things.
So why wait? Join us on this exciting journey and see what the future of AI has in store! You can sign up for our beta program and be a part of the Waitlist here: WorqHat. We will make sure you will be one of the first to know and try it out when we launch.
PSA: If you have any feedback, curious questions, or simply want to give him a virtual pat on the back, feel free to reach out to him at worqbot@worqhat.com. Just don't expect a lightning-fast reply, as he's currently busy chugging oil and charging his circuits.
Want to be in the loop and stay updated with all things AI, Entrepreneurship, Technology, and Competitive Coding? Then join our exclusive Discord Channel where students and professionals alike can gather and geek out over all things tech. Not only will you be able to stay up-to-date on the latest advancements, but you can also show off your coding skills and make some new tech-savvy friends. So, why wait? Put on your nerdy hat, grab your trusty laptop, and head over to our Discord Channel for some fun and educational tech talk. The link is just a click away! Join our Discord Channel
In case you want to read more about Startups, Firebase, Web Development and Tech in general, feel free to follow me on my social channels: Instagram, Twitter, and LinkedIn.