Replies: 22 comments 38 replies
-
|
I think in the end, multible chatGPT instances that all remember a small aspect and are managed in a tree like way might be needed for the project memory. Some that remember the files and have tools like a hardcoded thing for the programm names. Some that remember the overall project. Some that remember sertain parts of it. Some that decide how many layers it needs (depening on complexity and size of the project). Some that keep track of the problems or on what tasks it works. Since haing a plan and executing milestones helps to break down complex stuff. It should be 100% possible already just needs the work and knowledge on how to do it and what can go wrong while doing it and the cooperation. I think that is where it will fail (hopefully not). AutoGPT is amazing it was super fun working with it. If it had better memory(not perfect) and remembered how to remember or how to use notes and so on then it would be perfect, But that seems to be not only chatGPT's problem but also AutoGPT but im encouraged by recent attemtps and focus on memory and trying to involve the community and making things more fun so people are not bored (for those that easily are while the great not bored people play to those strengts. Alignment of interessts will also be important. Centralizing power in peoples hands that have a insentive to not get to X because they would be less important is also a risk.) My subjective viewpoint: If the backend is good enought then no reason to keep them but i suggest integrating maybe pinecone (maybe redis if there is a advantague) into your own custom JSONFileMemory if helpful altho try to stay independent by having a few independent systems that are joint in a tree like way or automatic selection maybe even a bit of simple machine lerning to pick the right thing in there a possibility. or just focus on a simple hardcoded thing to have gpt cycle through something to remember what it did, but some lerning system seems most exiting but it would have to realize many things or it risks going down on a missed level of understanding/premise. Like for example a not full understanding of how GPT does things or language models do certain things and what is helpful and what is not my a slight overantromorphising |
Beta Was this translation helpful? Give feedback.
-
|
I see that you have implanted this already but for what it's worth I think it is much better now. Makes it clear what is what. |
Beta Was this translation helpful? Give feedback.
-
|
My understanding of using backends like embedded Weaviate in AutoGPT was that they would facilitate the ability for the system to iterate on certain tasks across multiple runs of AutoGPT, such as integrating and utilizing particular APIs, and adopting best practices that might have been incorporated into code produced by AutoGPT. Using this long-term memory feature, subsequent AutoGPT instances could quickly recall past interactions with APIs or code snippets, allowing for more efficient and streamlined development. This is because we could presume that AutoGPT would grasp the concept rapidly in a similar manner as before, with far less token burn, thereby enabling us to focus on new tasks that could be part of a larger objective. To manage this process, I had the idea of snapshotting the Weaviate instance in tandem with a journal of AutoGPT AI settings. This would allow us to revert to a specific point in time if the introduction of new data into the system resulted in a regression in progress. With a single larger objective in focus, this setup would support an AutoGPT instance that's specialized in tasks it has been trained for, leveraging the embedded Weaviate memory. However, I've been wrestling with some questions. Are my assumptions completely off the mark? Does switching off these memory models represent a step backward in the concept of long-term memory? Is there more depth to this JSON model that I may not fully comprehend? Could simply carrying forward the JSON memory achieve the same objective? I'm hoping and assuming it's the latter. I am hoping that in addition to a simpler long term memory scheme, we might have the ability to pre-seed, combine the learnings of two separate AutoGPT instances with distinct specializations (i.e. marketing strategy and creative development) and put them to work together, and possibly even create some code that can remove or modify the JSON memory in creative ways, sort of like Crispr for AI. crispr_memai.py ("remove all knowledge and learned skills related to deprecated function oauth2client") It would be nice to start burning some local resource crunching and moving giant JSON files into memory whilst the little Thinking spinner does work. Any insights or clarifications appreciated. |
Beta Was this translation helpful? Give feedback.
-
|
I have difficulty following everything you say but moving away from the local JSON long-term solution is a bad idea. First, not everyone has access to what you speak of. Seconds of all, no matter how fancy other remote memory solutions are, I think it doesn't matter that much right now. A reliable way of feeding memory is first needed and there isn't one. Sometimes my Auto-GPT moves on just fine but as soon as little context remains in STM it's game over. There should be a part of the STM reserve for context on what it's currently working on. And instruction to ask for data from the LTM, this almost never happens. Even if all the tests that exist say it's all working it's not. |
Beta Was this translation helpful? Give feedback.
-
|
The primary purpose of vector memory has been to improve long-term performance of Auto-GPT. Persisting knowledge across multiple runs was not a part of that. Right now we're working on the primary purpose again, building a memory (or "retrieval augmentation") system that enhances performance during a run of arbitrary length. Once that works, we can look at sharing memory and persistence of knowledge across runs. |
Beta Was this translation helpful? Give feedback.
-
|
I'm a newbie so I'm not sure I'm understanding the question correctly. It seems that the need for external, third-party memory systems would be self-evident: bogging down the local memory might not be a big deal to some users, but others of us prefer to offload that elsewhere. It seems to be a big step backwards to remove this functionality entirely. Or am I misunderstanding the decision that's been made? |
Beta Was this translation helpful? Give feedback.
-
|
Having no long-term memory at all doesn't make any sense. Is this the case? |
Beta Was this translation helpful? Give feedback.
-
|
They did not say they were removing long term memory completely, just dropping all but Redis as there's not enough time to keep up with all 4. Personally, I have been using Weaviate, but don't mind switching to Redis if it means more work can happen to improve Auto-GPT. |
Beta Was this translation helpful? Give feedback.
-
|
Small correction to @SaViGnAnO: we dropped everything but the "local" memory provider (now called |
Beta Was this translation helpful? Give feedback.
-
|
what is needed is a working implementation, irrespective of the technology stack. We ned to focus on features not technology. Once a single feature set is operational, we can decide to implement other technology stacks. The rework, if any, can be done within the community, once we have something to build upon. My biggest challenge has been that to do any real work that is more than "right now", we need some sort of context that is persistent. Writing files by command isn't really "persistent context" as much as it is "file cabinet storage on command." Memory, short- or long-term, needs to be something that the system can fall back to when it seems like it can't figure out a context. |
Beta Was this translation helpful? Give feedback.
-
Hoping this is what I've been conveying too as it's just how I think. Obviously, I don't mean literary, but in terms of
and the list goes on, tell me what you guys need and I will give you a solution for us. |
Beta Was this translation helpful? Give feedback.
-
|
I invite you to take a look at the Agent workflow v2 drawing board, linked in #3536. We're trying to address the exact issue you raise by making room for retrieval augmentation in the agent loop. |
Beta Was this translation helpful? Give feedback.
-
|
didn't you just describe how GIT works ? :D |
Beta Was this translation helpful? Give feedback.
-
Who? me or @Pwuts ? if me, perhaps I don't know 😎 I don't base my ideas on others, I use AI to come up with solutions from given input. |
Beta Was this translation helpful? Give feedback.
-
|
I'd like to advocate for the continued use of a vector database, specifically Pinecone or Weaviate, as optional memory backends for Auto-GPT. While I generally understand the challenges of maintaining compatibility with multiple backends, especially as their APIs evolve, I believe the benefits of a singular vector database for AI memory storage are significant. Efficiency:Both Pinecone and Weaviate are designed to handle vector data, which aligns well with the needs of AI. This could make them more efficient than the current JSON file memory system. They are built for tasks like these, which can streamline development and reduce token burn. Scalability:Vector databases could offer better scalability. In a scenario where multiple AIs want to write to the same memory at the same time, a vector database could handle this more gracefully than JSON files. Avoiding Reinvention:Leveraging tools that are already designed for this purpose could save time from reinventing the wheel. While modifying the JSON file memory system could work, using a vector database like Pinecone or Weaviate might be more efficient out of the box. From @ianbmacdonald's comment, vector databases could also support the idea of snapshotting the memory state along with Auto-GPT settings. While I'm arguing for Pinecone, it seems from the thread that Weaviate is also a popular choice. The idea is that having at least one vector database as a memory backend can bring significant benefits to Auto-GPT. I believe it's worth considering before removing all three. Obviously, these are just thoughts and I'm open to other ideas; just a couple extra cents. If they must go to allow focus elsewhere, then so be it. 🤷 |
Beta Was this translation helpful? Give feedback.
-
This is a very general statement, but I'm interested to hear what you mean. Can you provide some examples or hypothetical cases for which it would be true? FYI, all third-party memory provider implementations have already been removed, and adding them back would require some work because of changes to the memory storage interface. |
Beta Was this translation helpful? Give feedback.
-
|
Vector db is useful among other things for adding context to the task. For instance user might want to add a design document and then ask AutoGPT to produce something based on "rules" from that document. Perhaps that's not the same thing as what you use memory for, but its still a valid point for using vector db. |
Beta Was this translation helpful? Give feedback.
-
|
I have a simple question - With Redis backend enabled, even for the same queries, the software was repeating the entire steps. ie, whatever was learned and stored in the database was not utilized. Am I missing something ? My understand was the memory backend helps to build knowledge so that it can be used for future tasks. Is this the idea or something else ? |
Beta Was this translation helpful? Give feedback.
-
|
Vector memory was disabled in v0.3.1 because it didn't help performance. As of v0.4.0 (published just now), the memory storage system has been reworked but vector memory is not used yet. See #3536 for updates and the actual status on this. |
Beta Was this translation helpful? Give feedback.
-
|
Thanks @Pwuts
This is to say, when we are accessing the JSON TEXT file on the disc vs redis there is no difference and we are expecting see a better performance with a better performance memory and our performance metrics has proved this assumption. My original question was answered by you here:
My two cents : Is it necessary, ie is there a blocker if we don't have database ? Honestly what I have seen in last 2 decades is for most purposes something as simple as SQLite is super fast and with latest NVMe and Optane storage the disc access is extremely fast. The project in the meantime get the ability to get the workflow smooth for prompts, handle edge cases, time outs etc. Also always attempt to use products which can be self hosted and a permissible FOSS so that in the long term we don't end up in vendor locks. |
Beta Was this translation helpful? Give feedback.
-
|
Speed is almost never an issue with Auto-GPT since the all-dominating bottleneck is the turn-around time of the LLM. So while developing the memory system, we want to focus on the data structure, not the storage mechanism. Using a simple in-memory store with JSON persistence is the easiest way to do that and leave room for experimentation. |
Beta Was this translation helpful? Give feedback.
-
|
We have already put lots of effort to build the Pinecone memory knowledge. Now pinecone support has been removed from v0.4.0 Release. I think this is not the proper way to do. We can not transfer or download the memory to other system like Redis. |
Beta Was this translation helpful? Give feedback.
-
|
If you really want to, you can keep using an old version of Auto-GPT with the old memory implementation. We removed it because our evaluation showed it didn't enhance performance overall, so it had little added value to keep the old implementation in place. Support for Pinecone will most probably be added back in the future, either in the main codebase or as a plugin. For now, having only one memory backend to support allows us to focus on memory functionality instead of compatibility. |
Beta Was this translation helpful? Give feedback.
-
|
I guess having an abstract class of memory storage should be enough, who wants, can always implement own solution. |
Beta Was this translation helpful? Give feedback.
-
|
Langchain recently added support for Vectara. They seem to be the new guy on the block. I haven't had much experience with other memory solutions like Milvus or Weviate but compared with Pinecone, Vectara supposedly has a lot of things automated like splitting up documents and embedding them (we don't have to pay for text-embedding-ada-002 to embed documents anymore) and Neural Search (not sure how accurate or effective it is at retrieving documents though). I'm working on a project myself to give persistent long-term memory capabilities to language models like GPT-4 so it can index a vector database with content such as books, research papers, articles, social media content, etc, update it on the fly with new, relevant information and query it across multiple different instances and runs. I was a bit disappointed actually that the memory capabilities have been redacted from Autogpt. What is the reason for it? Can someone explain? Sorry, but I haven't caught up with all the updates on AutoGPT. It also seems like progress is slowing down too. I remember in the first month, Autogpt had like an update every other day, but now it seems like once a month. What happened? |
Beta Was this translation helpful? Give feedback.
-
Just scroll through this page.
We're merging fewer small changes and more very big changes, most of which are still in the works. We're also trying to get to a more regular release schedule and a more effective workflow to process incoming contributions. |
Beta Was this translation helpful? Give feedback.
-
|
Pinecone is fine with me! |
Beta Was this translation helpful? Give feedback.
-
Thanks for the great work. Regarding backend, maintaining Milvus support should make sense for further development of the project. I don't se e the sense of an AI saving storage locally. Milvus is opensource and can be accessed remotely even as selfhosted. I guess anybody who would like to build a proper AI should have a proper backend built, and again the only opensource vector database available to do the right job is Milvus. I strongly suggest to keep it. Thanks. |
Beta Was this translation helpful? Give feedback.
-
|
They can do it a separate module and it will be developed separately. Adding a specific interface and option for that in the core logic should be available in my opinion. |
Beta Was this translation helpful? Give feedback.
-
|
First, thanks for this project and thoughts on how to move this forward with useful features. This seems the right place to discuss my experience with Auto-GPT. Presently, I am trying to work with AI to create engaging long fiction. The memory for this type of work is insufficient and results in the AI asking about things we have already moved past. I've been poking at different extensions for longer memory but none of them seem able to do this. For instance, we work on Chapter 1, then refer to its contents in Chapter 5, which the system doesn't understand. Am I approaching this the wrong way? Fyi, I have gotten a few noticeably short stories out of it that make sense but are not production worthy. I'm happy to share some results if requested. 🖖😎🤓 |
Beta Was this translation helpful? Give feedback.
-
|
We know we can assign AutoGPT short-term, quickly executable goals as well as larger long-term goals which take AutoGPT & AI a longer period of time (education period/process/cycle/term ) to fully achieve/execute. During the education period/process/cycle/term for a long-term goal (prior to the long-term goal being fully achieved/executed); does AutoGPT have any internal triggers/policies that (a) force or automatically reset/purge/restart the memory index (both short term and long term memory), and (b) overwrite the namespace, at any stage during the long-term goal education period if a certain action/error occurs? Example A: If the user reboots his/her local device (connected to AutoGPT), does AutoGPT automatically reset/purge/restart the memory index? In the same scenario, does AutoGPT overwrite the namespace? Example B: If AutoGPT fails to educate/execute a command (i.e. cannot handle command error) or faces another error that brings the process to a halt – i.e forcing the user to instruct a restart process command, does AutoGPT automatically reset/purge/restart the memory index? In the same scenario, does AutoGPT overwrite the namespace? |
Beta Was this translation helpful? Give feedback.
-
|
I vote for waviate if we had to choose one. It's very easy to set up in a docker, because they have a docker builder app on their website. It can run just a memory back end, or download some engines itself and preprocess with GPU support. Its api seems feature rich. |
Beta Was this translation helpful? Give feedback.
-
|
Does Waviate cover the best ambitions for this project based on their use cases: I'd like to be able to run everything locally with my hardware for most creative projects. |
Beta Was this translation helpful? Give feedback.
-
|
I have been running Auto-GPT with weaviate+transformers memory and tried different locally run models like GPT4All and a few others. It sort of works, but the open source models maily use CPU, which is ridicolously slow, and in several cases too limited context to perform a Auto-GPT action choice, especially if plugins or local commands are used. GPT4All has the ability to also use NVidia CUDA, which helps, but you'd have to write an api on top to let Auto-GPT use it. I found a few ways tp onnect it all without using OpenAI, but the execution tme and the grenerated responses was far from good enough. I am currently working on creating a guide on how to convert open source models to different formats in a way that utilizes the CPU and both the Nvidia and the integrated Intel graphics, wile down-quantizing them to run efficiently as a basic GPT4All cpu setup takes 5+ minutes to generate a response. |
Beta Was this translation helpful? Give feedback.
-
|
I’ve tried it on a local docker with and without bits own Ai model and it seems to work great!
Sent from Outlook for iOS<https://aka.ms/o0ukef>
…________________________________
From: Scott Cole ***@***.***>
Sent: Saturday, June 17, 2023 1:25:20 AM
To: Significant-Gravitas/Auto-GPT ***@***.***>
Cc: Phil ***@***.***>; Comment ***@***.***>
Subject: Re: [Significant-Gravitas/Auto-GPT] ❓Looking for opinions: Added value of different memory backends (Discussion #4280)
Does Waviate cover the best ambitions for this project based on their use cases:
https://weaviate.io/developers/weaviate/more-resources/example-use-cases#:~:text=With%20Weaviate%20and%20its%20Contextionary,Product%20search%20for%20E%2Dcommerce.
I'd like to be able to run everything locally with my hardware for most creative projects.
—
Reply to this email directly, view it on GitHub<#4280 (reply in thread)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ALCDNRJ3BPJET7LQALSAUZDXLR6WRANCNFSM6AAAAAAYF3XEIU>.
You are receiving this because you commented.Message ID: ***@***.***>
|
Beta Was this translation helpful? Give feedback.
-
|
I'm also a bit excited by its potential when run with haystack |
Beta Was this translation helpful? Give feedback.
-
|
To me, the fundamental problem is that regardless of memory back-end we can't get around the amount of 'active' memory and the token size. I think we're doing a disservice to the conversation by even calling it 'memory'. We have information that is weighted but to me the primary issue is the limit to the amount of information we have available at any given time. As such, this conversation seems to be trying to solve a problem way ahead of it's time. What is really required is an better method to take the 'memory', understand the context and application, and then break down the task at hand into problems that can reasonably be handled by the 'tokens' available for that query. Auto-gpt seems to be pointing the same approach to solving the limitations in GPT-4 that GPT-4 itself has. The 'memory' back-end doesn't matter much. A simplistic example with chatgpt plugins... have chatgpt try to perform a task while logging all the output to a noteable notebook. Then have another chatgpt with a goal of helping a chatgpt agent perform the task read the notebook. I'm not suggesting this is analogous to auto-gpt, but it's a really good way to get some insight. There is a finite and very small limit to what one query can accomplish, so we need to break down the queries to tiny bites (that would need to include the relevant information from 'memory). You can optimize the memory all you want but someone or someAI (tm) will have to extrapolate that ask into tasks that are small enough to be solved in the tokens available. Access to massive amounts of background information doesn't help. I've got a ton of questions from here but I'm interested in the response here first. We're way behind big tech in terms of funding research but we also think from a different perspective... |
Beta Was this translation helpful? Give feedback.
-
|
You make good points.
This is pretty much what we're trying to do by integrating task management with a plan-execute cycle, where the user objectives are turned into an itemized task list, each of which can then be executed by a sub-agent or "worker". This helps both the scalability of Auto-GPT's problem solving capabilities and the possibilities for "memory" or "retrieval augmentation". |
Beta Was this translation helpful? Give feedback.
-
|
Dear Auto-GPT Team, I trust this message finds you well. I am reaching out to express my concerns regarding the recent decision to discontinue support for certain memory backends, including Weaviate, within the Auto-GPT project. At present, I am in the process of developing a suite of agents, each with specialized roles, to continuously monitor and suggest improvements. Concurrently, I am training a series of locally run, specialized open-source models to use Auto-GPT without the need for the OpenAI API. Over the past month, I have found the Weaviate backend to be an invaluable resource. It has enabled me to generate embeddings, thus avoiding the costs associated with using the OpenAI Ada model. This is particularly significant for my future work, as creating embeddings for the Auto-GPT source code alone incurs a cost of around $0.5. Upon the approval of my OpenAI-plugin developer application, I had planned to create a similar long-term memory plugin for ChatGPT. This plugin would have the capability to specify endpoints, thus enabling a connection to a private or publicly available Weaviate instance. This would allow ChatGPT Plus + plugins to access the same storage used by Auto-GPT, and facilitate user collaboration to build Weaviate knowledge databases on a variety of topics, accessible to both Auto-GPT and ChatGPT. The decision to discontinue Weaviate has led me to pause and reassess my plans. My potential alternatives would be to either fork this repository and proceed independently, or to start from scratch and build my own alternative using LangChain or a similar tool. I believe that JSON storage is not a practical solution if the objective is to create embeddings for an entire company's codebase and wiki, which includes hundreds of projects and requires traversing gigabytes of JSON data. The notebook plugin from ChatGPT, as suggested, would also be too restrictive for real-world business scenarios where the volume of data is substantial. This decision has left me deeply disappointed, as it disrupts several of my ongoing and future projects. The potential of Auto-GPT was the primary reason I dedicated my spare time over the past month to focus on it, and I regret not being more active here previously. I kindly request you to reconsider this decision. In my opinion, the ability to integrate with platforms like Weaviate and Pinecone is one of the greatest advantages of Auto-GPT in a business setting. If the team is unable to allocate time to this integration, I am more than willing to contribute my time and expertise. While my Python skills may not be exceptional, I have over 20 years of experience in developing business software and am confident that I can provide valuable input. Thank you for your consideration. Best regards, |
Beta Was this translation helpful? Give feedback.
-
|
I concur with a lot of this from a product design standpoint. Weaviate has great documentation, an online configurator, a huge number of potential use cases, can run local LLM's for preprocessing data, local or cloud instance, haystack framework etc etc, |
Beta Was this translation helpful? Give feedback.
-
|
The decision was not to discontinue support for Weaviate or any of the other backends, but to remove them for now to reduce the weight of the vector memory module while we work on a new memory implementation. Once we have a new memory implementation that works well, there's a good chance support for these vector store providers will return. About using Weaviate vs OpenAI for generating embeddings: I can't find the relevant information source right now, but as far as I'm aware the performance of If you have a good comparison of the performance of Weaviate's embedding engine vs OpenAI's |
Beta Was this translation helpful? Give feedback.
-
|
Hi all – I'm affiliated with Weaviate, and I've been made aware of this discussion just recently. TL;DR: we are happy to help and contribute For what it's worth, we do see quite some people use Weaviate with AutoGPT, so I can understand that some people would prefer to keep the backends in.
This is something we see more often in other OSS projects and frameworks. What's most commonly done is this: when somebody raises an issue on GH related to -in this case- Weaviate, the team tags one of the Weaviate contributors (or labels it) to fix it. In the end, it's an OSS community, and we can all chip in to help. Removing it might “risk” forking, leading to out-of-sync repos, etc. |
Beta Was this translation helpful? Give feedback.
-
|
Cool! Which email should I use to reach you? The one on your profile? |
Beta Was this translation helpful? Give feedback.
-
|
reinier [dot] vanderleer [at] agpt.co |
Beta Was this translation helpful? Give feedback.
-
|
Youve got mail |
Beta Was this translation helpful? Give feedback.
-
|
Hi! Any idea when you're planning to add support for more memory backends? |
Beta Was this translation helpful? Give feedback.
-
|
Would be great to have an update on this. Thanks! |
Beta Was this translation helpful? Give feedback.
-
|
I'm not one of the coders/contributors. Please stop CCing me.
Sincerely,
Scott Swain
https://OceanMedia.net
Sent with [Proton Mail](https://proton.me/) secure email.
…------- Original Message -------
On Monday, October 16th, 2023 at 09:37, amirvenus ***@***.***> wrote:
Would be great to have an update on this. Thanks!
—
Reply to this email directly, [view it on GitHub](#4280 (reply in thread)), or [unsubscribe](https://github.com/notifications/unsubscribe-auth/AFBST3KET5GN3QLZNE4N5LDX7VBCHAVCNFSM6AAAAAAYF3XEIWVHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM3TEOJTGU2DA).
You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>
|
Beta Was this translation helpful? Give feedback.
-
|
You have to unsubscribe by clicking Unsubscribe here: |
Beta Was this translation helpful? Give feedback.
-
|
@Pwuts I can see no vector database is used, Is there any plan of integrating any vectordb with AutoGPT? |
Beta Was this translation helpful? Give feedback.
-
|
Would you mind sharing your use case? |
Beta Was this translation helpful? Give feedback.
-
|
Sure, For now I was thinking about creating a discord bot for performing daily summarization report for different channels. |
Beta Was this translation helpful? Give feedback.
-
|
What do you think, Is it possible to add vectorDB or I can integrate and raise a PR whatever works? |
Beta Was this translation helpful? Give feedback.
-
|
Still curious why people not using Milvus vector database ? |
Beta Was this translation helpful? Give feedback.
and others
-
As we work on the memory system with the core team, we are eager to drop support for memory backends that do not add significant value over the base
JSONFileMemoryimplementation.The most recent release of Auto-GPT has support for the following memory stores:
LocalCache(will be renamed toJSONFileMemory)Support for Milvus, Pinecone and Weaviate will be removed as a part of our re-architecture effort because we have limited time, and keeping up support for those backends would slow us down significantly.
If you think Milvus, Pinecone or Weaviate should be supported again in the future, please let us know why! :) If the added value is clear, we will be happy to put some work into it.
Beta Was this translation helpful? Give feedback.
All reactions