Originally published at: Build Your First Human-in-the-Loop AI Agent with NVIDIA NIM | NVIDIA Technical Blog
AI agents powered by large language models (LLMs) help organizations streamline and reduce manual workloads. These agents use multilevel, iterative reasoning to analyze problems, devise solutions, and execute tasks with various tools. Unlike traditional chatbots, LLM-powered agents automate complex tasks by effectively understanding and processing information. To avoid potential risks in specific applications, maintaining human…
I was very excited to try out the models hosted on the Nvidia NIMs but the blog step by step walk thrught is not beginner friendly for example the state class for the StateGraph is not defined and it would take someone familiar with building agents with langgraph to know to also create a state for the agent.
Other than that, it’s a really nice post. I’m going to complete it’s implementation using the knowledge of langgraph and I’m looking forward to testing out the human in the loop component.
Hello, Thank you so much for trying this out. here is the link to the entire source code in jupyter notebook format GenerativeAIExamples/RAG/notebooks/langchain/NIM_tool_call_HumanInTheLoop_MultiAgents.ipynb at main · NVIDIA/GenerativeAIExamples in the notebook it has the langGraph state definiton and all other pieces that might be missed in the technical blog. Please note that the python environment build can be found here GenerativeAIExamples/RAG/notebooks at main · NVIDIA/GenerativeAIExamples. Hopefully it can help you get started. Please do ask more questions if you have any and we always welcome feedbacks.
Hi @zcharpy ,
This blog really seems helpful! I have tried the technical blog of AI Agent. I am currently trying to execute the final output when i select option 1 and confirm as y i am often been prompted with this error message could you please help, awaiting for your response
also as recommended in the github notebook text to imageis unable at the moment so i tried with other NIMs that support text to image but i am unable to use it
“https://ai.api.nvidia.com/v1/genai/briaai/bria-2.3”
that’s odd, could you add one more above line 18 and print the state , then copy and paste the result here for me to help you debug this?
@zcharpy : it seems to be working fine now and I am able to get the Content Creator part as output, however I am facing issues with Digital Artist, i see the NIM is not available in build.nvidia.com, could you suggest something on this please…
@zcharpy : Could you please help me with the Digital Artist part, I don’t see the availability of model on build.nvidi.com. Looking forward to seek your help on same, awaiting for your response.
Can you check if you can see this one stable-diffusion-xl Model by Stability AI | NVIDIA NIM ? this is where stable diffusion xl located
@zcharpy : As advised I have tried to use stable-diffusion-xl Model by Stability AI | NVIDIA NIM API key and tried, but I am being prompted with below error. I have tried to regenerate a new API key as well but still facing same issue, please check screenshots. I am able to get output only for Content generation
could you check if you have enough credits on NGC ? in my situation, there is 0 credits left , what is yours ?
I am passing llama model API key at beginning and later passing stable diffusion model API key , pleas check and help me
Thank you very much for being patient.
Can you do one more thing for me, when you re-generate your API key, choose 'Generate Personal Key ’ > add in the following scope Public API Endpoints, and Cloud Functions. then use the new API key to run through this again ? in the mean while i will pass along your error tracing to internal API folks and see if they know why your account cannot use the service when you still have credits left. It might take a bit time ( a day or 2 ) but i will let you know what they say.
@zcharpy :I don’t see an option anywhere as “Generate Personal Key” rather when i click on Get API key option I am been prompted with this dialog because where I see the API key gets generated and I can copy paste it and use further
I need to check with the NGC team how they implement this button in the backend, you must have cloud function and public ai endpoint enabled as a scope when you generate the API key for this model to work, I am not sure how they implement this. usually I login through this url Dashboard | NVIDIA NGC and then generate the key instead of the button on the UI.
I think this might be a bug, I am reporting this now to the NGC team.
It might take a few days. Apologies first for the delay…
@zcharpy :Sure i shall wait to hear back from you as I am not sure how do i proceed on this and get this fixed. It is strange though that I am able to access the API key for llama model but unfortunately same does not work for stable diffusion model although I have credits left. I will wait until you get the correspondence from NGC team, please keep me posted.
hello again, could you try replacing the YOUR_API_KEY with your key in below script and open a terminal and run the below command and paste back a screenshot of what happens –
invoke_url=‘https://ai.api.nvidia.com/v1/genai/stabilityai/stable-diffusion-xl’
authorization_header=“Authorization: Bearer YOUR_API_KEY”
accept_header=‘Accept: application/json’
content_type_header=‘Content-Type: application/json’
data=‘{
“text_prompts”: [
{
“text”: “underwater world, plants, shells, creatures, high detail, sharp focus, 4k”,
“weight”: 1
},
{
“text”: “” ,
“weight”: -1
}
],
“cfg_scale”: 5,
“sampler”: “K_DPM_2_ANCESTRAL”,
“seed”: 0,
“steps”: 25
}’
response=$(curl --silent -i -w “\n%{http_code}” --request POST
–url “$invoke_url”
–header “$authorization_header”
–header “$accept_header”
–header “$content_type_header”
–data “$data”
)
http_code=$(echo “$response” | tail -n 1)
echo “$response” | awk ‘/{/,EOF-1’
btw your API key is visible in the error trace screenshot so you should probably rotate the key ( delete this key and regenerate new key ) as to avoid other people who also see this thread to use your key :) just trying to look out for you.
@zcharpy : I have ran the code provided by you in VS code editor and generated a new API key, below is the response i am getting after executing. However I was getting error this part, please refer screenshot and help me
data = {
“text_prompts”: [
{
“text”: “A futuristic cyberpunk city with neon lights and flying cars”,
“weight”: 1
}
],
“cfg_scale”: 7,
“sampler”: “DPM++ 2M Karras”, # This line needed to be rectified
“seed”: 42,
“steps”: 50
}
So i updated it the code this way :
data = {
“text_prompts”: [
{
“text”: “A futuristic cyberpunk city with neon lights and flying cars”,
“weight”: 1
}
],
“cfg_scale”: 7,
“sampler”: “K_DPM_2_ANCESTRAL”, # This was later updated
“seed”: 42,
“steps”: 50
}
Output using sampler": "DPM++ 2M Karras
Output using sampler": K_DPM_2_ANCESTRAL
