[Doc] Generate of tutorial (#3230)

# Description

Generate of tutorial

# All Promptflow Contribution checklist:
- [ ] **The pull request does not introduce [breaking changes].**
- [ ] **CHANGELOG is updated for new features, bug fixes or other
significant changes.**
- [ ] **I have read the [contribution guidelines](../CONTRIBUTING.md).**
- [ ] **Create an issue and link to the pull request to get dedicated
review from promptflow team. Learn more: [suggested
workflow](../CONTRIBUTING.md#suggested-workflow).**

## General Guidelines and Best Practices
- [ ] Title of the pull request is clear and informative.
- [ ] There are a small number of commits, each of which have an
informative message. This means that previously merged commits do not
appear in the history of the PR. For more information on cleaning up the
commits in your PR, [see this
page](https://github.com/Azure/azure-powershell/blob/master/documentation/development-docs/cleaning-up-commits.md).

### Testing Guidelines
- [ ] Pull request includes test coverage for the included changes.

.github/workflows/samples_getstarted_flowasfunction.yml

@@ -9,7 +9,7 @@ on:
     - cron: "17 21 * * *" # Every day starting at 5:17 BJT
   pull_request:
     branches: [ main ]
-    paths: [ examples/tutorials/get-started/**, examples/flows/standard/web-classification/**, examples/flows/chat/chat-basic/**, .github/workflows/samples_getstarted_flowasfunction.yml, examples/requirements.txt, examples/connections/azure_openai.yml ]
+    paths: [ examples/tutorials/get-started/**, examples/*requirements.txt, .github/workflows/samples_getstarted_flowasfunction.yml ]
   workflow_dispatch:
 
 env:

.github/workflows/samples_getstarted_quickstart.yml

@@ -9,7 +9,7 @@ on:
     - cron: "55 21 * * *" # Every day starting at 5:55 BJT
   pull_request:
     branches: [ main ]
-    paths: [ examples/tutorials/get-started/**, examples/requirements.txt, examples/flows/standard/web-classification/**, examples/flows/evaluation/eval-classification-accuracy/**, .github/workflows/samples_getstarted_quickstart.yml, examples/connections/azure_openai.yml ]
+    paths: [ examples/tutorials/get-started/**, examples/*requirements.txt, .github/workflows/samples_getstarted_quickstart.yml ]
   workflow_dispatch:
 
 env:

.github/workflows/samples_getstarted_quickstartazure.yml

@@ -9,7 +9,7 @@ on:
     - cron: "24 20 * * *" # Every day starting at 4:24 BJT
   pull_request:
     branches: [ main ]
-    paths: [ examples/tutorials/get-started/**, examples/requirements.txt, examples/flows/standard/web-classification/**, examples/flows/evaluation/eval-classification-accuracy/**, .github/workflows/samples_getstarted_quickstartazure.yml, examples/connections/azure_openai.yml ]
+    paths: [ examples/tutorials/get-started/**, examples/*requirements.txt, .github/workflows/samples_getstarted_quickstartazure.yml ]
   workflow_dispatch:
 
 env:

.github/workflows/samples_runflowwithpipeline_pipeline.yml

@@ -9,7 +9,7 @@ on:
     - cron: "33 20 * * *" # Every day starting at 4:33 BJT
   pull_request:
     branches: [ main ]
-    paths: [ examples/tutorials/run-flow-with-pipeline/**, examples/flows/standard/web-classification/**, .github/workflows/samples_runflowwithpipeline_pipeline.yml, examples/requirements.txt, examples/connections/azure_openai.yml ]
+    paths: [ examples/tutorials/run-flow-with-pipeline/**, examples/*requirements.txt, .github/workflows/samples_runflowwithpipeline_pipeline.yml ]
   workflow_dispatch:
 
 env:

.github/workflows/samples_runmanagement_cloudrunmanagement.yml

@@ -9,7 +9,7 @@ on:
     - cron: "24 20 * * *" # Every day starting at 4:24 BJT
   pull_request:
     branches: [ main ]
-    paths: [ examples/tutorials/run-management/**, examples/requirements.txt, examples/flows/standard/web-classification/**, .github/workflows/samples_runmanagement_cloudrunmanagement.yml, examples/connections/azure_openai.yml ]
+    paths: [ examples/tutorials/run-management/**, examples/*requirements.txt, .github/workflows/samples_runmanagement_cloudrunmanagement.yml ]
   workflow_dispatch:
 
 env:

.github/workflows/samples_runmanagement_runmanagement.yml

@@ -9,7 +9,7 @@ on:
     - cron: "51 20 * * *" # Every day starting at 4:51 BJT
   pull_request:
     branches: [ main ]
-    paths: [ examples/tutorials/run-management/**, examples/requirements.txt, examples/flows/standard/web-classification/**, .github/workflows/samples_runmanagement_runmanagement.yml, examples/connections/azure_openai.yml ]
+    paths: [ examples/tutorials/run-management/**, examples/*requirements.txt, .github/workflows/samples_runmanagement_runmanagement.yml ]
   workflow_dispatch:
 
 env:

.github/workflows/samples_tracing_autogengroupchat_traceautogengroupchat.yml

@@ -9,7 +9,7 @@ on:
     - cron: "11 20 * * *" # Every day starting at 4:11 BJT
   pull_request:
     branches: [ main ]
-    paths: [ examples/tutorials/tracing/autogen-groupchat/**, .github/workflows/samples_tracing_autogengroupchat_traceautogengroupchat.yml, examples/requirements.txt, examples/connections/azure_openai.yml ]
+    paths: [ examples/tutorials/tracing/autogen-groupchat/**, examples/*requirements.txt, .github/workflows/samples_tracing_autogengroupchat_traceautogengroupchat.yml ]
   workflow_dispatch:
 
 env:

.github/workflows/samples_tracing_customotlpcollector_otlptracecollector.yml

@@ -9,7 +9,7 @@ on:
     - cron: "22 21 * * *" # Every day starting at 5:22 BJT
   pull_request:
     branches: [ main ]
-    paths: [ examples/tutorials/tracing/custom-otlp-collector/**, .github/workflows/samples_tracing_customotlpcollector_otlptracecollector.yml, examples/requirements.txt, examples/connections/azure_openai.yml ]
+    paths: [ examples/tutorials/tracing/custom-otlp-collector/**, examples/*requirements.txt, .github/workflows/samples_tracing_customotlpcollector_otlptracecollector.yml ]
   workflow_dispatch:
 
 env:

.github/workflows/samples_tracing_langchain_tracelangchain.yml

@@ -9,7 +9,7 @@ on:
     - cron: "21 19 * * *" # Every day starting at 3:21 BJT
   pull_request:
     branches: [ main ]
-    paths: [ examples/tutorials/tracing/langchain/**, .github/workflows/samples_tracing_langchain_tracelangchain.yml, examples/requirements.txt, examples/connections/azure_openai.yml ]
+    paths: [ examples/tutorials/tracing/langchain/**, examples/*requirements.txt, .github/workflows/samples_tracing_langchain_tracelangchain.yml ]
   workflow_dispatch:
 
 env:

.github/workflows/samples_tracing_llm_tracellm.yml

@@ -9,7 +9,7 @@ on:
     - cron: "36 21 * * *" # Every day starting at 5:36 BJT
   pull_request:
     branches: [ main ]
-    paths: [ examples/tutorials/tracing/llm/**, .github/workflows/samples_tracing_llm_tracellm.yml, examples/requirements.txt, examples/connections/azure_openai.yml ]
+    paths: [ examples/tutorials/tracing/llm/**, examples/*requirements.txt, .github/workflows/samples_tracing_llm_tracellm.yml ]
   workflow_dispatch:
 
 env:

docs/tutorials/index.md

@@ -2,31 +2,38 @@
 
 This section contains a collection of flow samples and step-by-step tutorials.
 
-|Area|<div style="width:250px">Sample</div>|Description|
+|Category|<div style="width:250px">Sample</div>|Description|
 |--|--|--|
-|SDK|[Getting started with prompt flow](https://github.com/microsoft/promptflow/blob/main/examples/tutorials/get-started/quickstart.ipynb)| A step by step guidance to invoke your first flow run.
-|CLI|[Chat with PDF](https://github.com/microsoft/promptflow/blob/main/examples/tutorials/e2e-development/chat-with-pdf.md)| An end-to-end tutorial on how to build a high quality chat application with prompt flow, including flow development and evaluation with metrics.
-|SDK|[Chat with PDF - test, evaluation and experimentation](https://github.com/microsoft/promptflow/blob/main/examples/flows/chat/chat-with-pdf/chat-with-pdf.ipynb)| We will walk you through how to use prompt flow Python SDK to test, evaluate and experiment with the "Chat with PDF" flow.
-|SDK|[Connection management](https://github.com/microsoft/promptflow/blob/main/examples/connections/connection.ipynb)| Manage various types of connections using sdk
-|CLI|[Working with connection](https://github.com/microsoft/promptflow/blob/main/examples/connections/README.md)| Manage various types of connections using cli
-|SDK|[Run prompt flow in Azure AI](https://github.com/microsoft/promptflow/blob/main/examples/tutorials/get-started/quickstart-azure.ipynb)| A quick start tutorial to run a flow in Azure AI and evaluate it.
-|SDK|[Flow run management in Azure AI](https://github.com/microsoft/promptflow/blob/main/examples/tutorials/run-management/cloud-run-management.ipynb)| Flow run management in azure AI
-|AZURE|[Develop promptflow copilot](https://github.com/microsoft/promptflow/blob/main/examples/tutorials/develop-promptflow-copilot/develop-promptflow-copilot.md)| A step by step guidance to develop a promptflow copilot.
-
-## Samples
-
-|Area|<div style="width:250px">Sample</div>|Description|
-|--|--|--|
-|Standard Flow|[basic](https://github.com/microsoft/promptflow/tree/main/examples/flows/standard/basic)| a basic flow with prompt and python tool.
-|Standard Flow|[basic-with-connection](https://github.com/microsoft/promptflow/tree/main/examples/flows/standard/basic-with-connection)| a basic flow using custom connection with prompt and python tool
-|Standard Flow|[basic-with-builtin-llm](https://github.com/microsoft/promptflow/tree/main/examples/flows/standard/basic-with-builtin-llm)| a basic flow using builtin llm tool
-|Standard Flow|[customer-intent-extraction](https://github.com/microsoft/promptflow/tree/main/examples/flows/standard/customer-intent-extraction)| a flow created from existing langchain python code
-|Standard Flow|[web-classification](https://github.com/microsoft/promptflow/tree/main/examples/flows/standard/web-classification)| a flow demonstrating multi-class classification with LLM. Given an url, it will classify the url into one web category with just a few shots, simple summarization and classification prompts.
-|Standard Flow|[autonomous-agent](https://github.com/microsoft/promptflow/tree/main/examples/flows/standard/autonomous-agent)| a flow showcasing how to construct a AutoGPT flow to autonomously figures out how to apply the given functions to solve the goal, which is film trivia that provides accurate and up-to-date information about movies, directors, actors, and more.
-|Chat Flow|[chat-with-wikipedia](https://github.com/microsoft/promptflow/tree/main/examples/flows/chat/chat-with-wikipedia)| a flow demonstrating Q&A with GPT3.5 using information from Wikipedia to make the answer more grounded.
-|Chat Flow|[chat-with-pdf](https://github.com/microsoft/promptflow/tree/main/examples/flows/chat/chat-with-pdf)| a flow that allow you to ask questions about the content of a PDF file and get answers.
-|Evaluation Flow|[eval-classification-accuracy](https://github.com/microsoft/promptflow/tree/main/examples/flows/evaluation/eval-classification-accuracy)| a flow illustrating how to evaluate the performance of a classification system.
-
-Learn more:  [Try out more promptflow examples.](https://github.com/microsoft/promptflow/tree/main/examples)
+|Tracing|[Tracing](https://github.com/microsoft/promptflow/blob/main/examples/tutorials/tracing/README.md)| Prompt flow provides the tracing feature to capture and visualize the internal execution details for all flows|
+|Tracing|[Tracing with llm application](https://github.com/microsoft/promptflow/blob/main/examples/tutorials/tracing/llm/trace-llm.ipynb)|Tracing LLM application|
+|Tracing|[Tracing with autogen](https://github.com/microsoft/promptflow/blob/main/examples/tutorials/tracing/autogen-groupchat/trace-autogen-groupchat.ipynb)|Tracing LLM calls in autogen group chat application|
+|Tracing|[Tracing with langchain apps](https://github.com/microsoft/promptflow/blob/main/examples/tutorials/tracing/langchain/trace-langchain.ipynb)|Tracing LLM calls in langchain application|
+|Tracing|[Tracing with custom opentelemetry collector](https://github.com/microsoft/promptflow/blob/main/examples/tutorials/tracing/custom-otlp-collector/otlp-trace-collector.ipynb)|A tutorial on how to levarage custom OTLP collector.|
+|Prompty|[Getting started with prompty](https://github.com/microsoft/promptflow/blob/main/examples/prompty/basic/prompty-quickstart.ipynb)|A quickstart tutorial to run a prompty and evaluate it.|
+|Prompty|[Chat with prompty](https://github.com/microsoft/promptflow/blob/main/examples/prompty/chat-basic/chat-with-prompty.ipynb)|A quickstart tutorial to run a chat prompty and evaluate it.|
+|Prompty|[Prompty output format](https://github.com/microsoft/promptflow/blob/main/examples/prompty/format-output/prompty-output-format.ipynb)||
+|Flow|[Getting started with flex flow in azure](https://github.com/microsoft/promptflow/blob/main/examples/flex-flows/basic/flex-flow-quickstart-azure.ipynb)|A quickstart tutorial to run a flex flow and evaluate it in azure.|
+|Flow|[Getting started with flex flow](https://github.com/microsoft/promptflow/blob/main/examples/flex-flows/basic/flex-flow-quickstart.ipynb)|A quickstart tutorial to run a flex flow and evaluate it.|
+|Flow|[Chat with class based flex flow in azure](https://github.com/microsoft/promptflow/blob/main/examples/flex-flows/chat-basic/chat-with-class-based-flow-azure.ipynb)|A quickstart tutorial to run a class based flex flow and evaluate it in azure.|
+|Flow|[Chat with class based flex flow](https://github.com/microsoft/promptflow/blob/main/examples/flex-flows/chat-basic/chat-with-class-based-flow.ipynb)|A quickstart tutorial to run a class based flex flow and evaluate it.|
+|Flow|[Stream chat with flex flow](https://github.com/microsoft/promptflow/blob/main/examples/flex-flows/chat-stream/chat-stream-with-flex-flow.ipynb)|A quickstart tutorial to run a class based flex flow in stream mode and evaluate it.|
+|Flow|[Run dag flow in azure](https://github.com/microsoft/promptflow/blob/main/examples/tutorials/get-started/quickstart-azure.ipynb)|A quickstart tutorial to run a flow in Azure AI and evaluate it.|
+|Flow|[Chat with pdf in azure](https://github.com/microsoft/promptflow/blob/main/examples/flows/chat/chat-with-pdf/chat-with-pdf-azure.ipynb)|A tutorial of chat-with-pdf flow that executes in Azure AI|
+|Flow|[Getting started with dag flow](https://github.com/microsoft/promptflow/blob/main/examples/tutorials/get-started/quickstart.ipynb)|A quickstart tutorial to run a flow and evaluate it.|
+|Flow|[Execute flow as a function](https://github.com/microsoft/promptflow/blob/main/examples/tutorials/get-started/flow-as-function.ipynb)|This guide will walk you through the main scenarios of executing flow as a function.|
+|Flow|[Run flows in azure ml pipeline](https://github.com/microsoft/promptflow/blob/main/examples/tutorials/run-flow-with-pipeline/pipeline.ipynb)|Create pipeline using components to run a distributed job with tensorflow|
+|Flow|[Flow run management in azure](https://github.com/microsoft/promptflow/blob/main/examples/tutorials/run-management/cloud-run-management.ipynb)|Flow run management in Azure AI|
+|Flow|[Flow run management](https://github.com/microsoft/promptflow/blob/main/examples/tutorials/run-management/run-management.ipynb)|Flow run management|
+|Flow|[Evaluate with langchain's evaluator](https://github.com/microsoft/promptflow/blob/main/examples/flex-flows/eval-criteria-with-langchain/langchain-eval.ipynb)|A tutorial to converting LangChain criteria evaluator application to flex flow.|
+|Flow|[Chat with pdf - test, evaluation and experimentation](https://github.com/microsoft/promptflow/blob/main/examples/flows/chat/chat-with-pdf/chat-with-pdf.ipynb)|A tutorial of chat-with-pdf flow that allows user ask questions about the content of a PDF file and get answers|
+|Deployment|[Deploy flow using kubernetes](https://github.com/microsoft/promptflow/blob/main/examples/tutorials/flow-deploy/kubernetes/README.md)| This example demos how to deploy flow as a Kubernetes app|
+|Deployment|[Deploy a flow using docker](https://github.com/microsoft/promptflow/blob/main/examples/tutorials/flow-deploy/docker/README.md)| This example demos how to deploy flow as a docker app|
+|Deployment|[Create service with flow](https://github.com/microsoft/promptflow/blob/main/examples/tutorials/flow-deploy/create-service-with-flow/README.md)| This example shows how to create a simple service with flow|
+|Deployment|[Distribute flow as executable app](https://github.com/microsoft/promptflow/blob/main/examples/tutorials/flow-deploy/distribute-flow-as-executable-app/README.md)| This example demos how to package flow as a executable app|
+|Deployment|[Deploy flow using azure app service](https://github.com/microsoft/promptflow/blob/main/examples/tutorials/flow-deploy/azure-app-service/README.md)| This example demos how to deploy a flow using Azure App Service|
+|Rag|[Tutorial: chat with pdf](https://github.com/microsoft/promptflow/blob/main/examples/tutorials/e2e-development/chat-with-pdf.md)| Retrieval Augmented Generation (or RAG) has become a prevalent pattern to build intelligent application with Large Language Models (or LLMs) since it can infuse external knowledge into the model, which is not trained with those up-to-date or proprietary information|
+|Rag|[Tutorial: how prompt flow helps on quality improvement](https://github.com/microsoft/promptflow/blob/main/examples/tutorials/flow-fine-tuning-evaluation/promptflow-quality-improvement.md)| This tutorial is designed to enhance your understanding of improving flow quality through prompt tuning and evaluation|
+|Rag|[Develop promptflow copilot](https://github.com/microsoft/promptflow/blob/main/examples/tutorials/develop-promptflow-copilot/develop-promptflow-copilot.md)| In this tutorial, we will provide a detailed walkthrough on creating a RAG-based copilot using the Azure Machine Learning promptflow toolkit|
 
 
+Learn more: [Try out more promptflow examples.](https://github.com/microsoft/promptflow/tree/main/examples)

examples/tutorials/develop-promptflow-copilot/develop-promptflow-copilot.md

@@ -1,3 +1,8 @@
+---
+category: rag
+weight: 30
+---
+
 # Develop promptflow copilot
 
 In this tutorial, we will provide a detailed walkthrough on creating a RAG-based copilot using the Azure Machine Learning promptflow toolkit. Our tutorial will cover a range of essential topics, including:

examples/tutorials/e2e-development/chat-with-pdf.md

@@ -1,5 +1,7 @@
 ---
 resources: examples/connections/azure_openai.yml, examples/flows/chat/chat-with-pdf
+cloud: local
+category: rag
 ---
 
 # Tutorial: Chat with PDF

examples/tutorials/flow-deploy/azure-app-service/README.md

@@ -1,5 +1,7 @@
 ---
 resources: examples/connections/azure_openai.yml, examples/flows/standard/web-classification
+category: deployment
+weight: 60
 ---
 
 # Deploy flow using Azure App Service

examples/tutorials/flow-deploy/create-service-with-flow/README.md

@@ -1,5 +1,7 @@
 ---
 resources: examples/tutorials/flow-deploy/create-service-with-flow
+category: deployment
+weight: 50
 ---
 
 # Create service with flow

examples/tutorials/flow-deploy/distribute-flow-as-executable-app/README.md

@@ -1,5 +1,7 @@
 ---
 resources: examples/connections/azure_openai.yml, examples/flows/standard/web-classification
+category: deployment
+weight: 50
 ---
 
 # Distribute flow as executable app

examples/tutorials/flow-deploy/docker/README.md

@@ -1,5 +1,7 @@
 ---
 resources: examples/connections/azure_openai.yml, examples/flows/standard/web-classification
+category: deployment
+weight: 40
 ---
 
 # Deploy a flow using Docker

examples/tutorials/flow-deploy/kubernetes/README.md

@@ -1,5 +1,7 @@
 ---
 resources: examples/connections/azure_openai.yml, examples/flows/standard/web-classification
+category: deployment
+weight: 30
 ---
 
 # Deploy flow using Kubernetes

examples/tutorials/flow-fine-tuning-evaluation/promptflow-quality-improvement.md

@@ -1,5 +1,6 @@
 ---
 resources: examples/connections/azure_openai.yml, examples/flows/chat/chat-basic, examples/flows/chat/chat-math-variant, examples/flows/evaluation/eval-chat-math
+category: rag
 ---
 
 # Tutorial: How prompt flow helps on quality improvement

examples/tutorials/tracing/README.md

@@ -1,5 +1,7 @@
 ---
 resources: examples/tutorials/tracing/
+cloud: local
+category: tracing
 ---
 
 ## Tracing

scripts/readme/ghactions_driver/readme_step.py

@@ -359,7 +359,7 @@ def write_workflow(
             from .resource_resolver import resolve_tutorial_resource
 
             path_filter = resolve_tutorial_resource(
-                workflow_name, readme_path.resolve()
+                workflow_name, readme_path.resolve(), output_telemetry
             )
         else:
             if (

scripts/readme/ghactions_driver/readme_templates/tutorial_index.md.jinja2

@@ -0,0 +1,10 @@
+# Tutorials
+
+This section contains a collection of flow samples and step-by-step tutorials.
+
+|Category|<div style="width:250px">Sample</div>|Description|
+|--|--|--|
+{% for item in items %}|{{ item.category }}|[{{ item.title }}]({{ item.url }})|{{ item.description }}|
+{% endfor %}
+
+Learn more: [Try out more promptflow examples.](https://github.com/microsoft/promptflow/tree/main/examples)