Introducing Gradio Clients
WatchIntroducing Gradio Clients
WatchIn this guide, we'll use YOLOv10 to perform real-time object detection in Gradio from a user's webcam feed. We'll utilize the latest streaming features introduced in Gradio 5.0. You can see the finished product in action below:
Start by installing all the dependencies. Add the following lines to a requirements.txt
file and run pip install -r requirements.txt
:
opencv-python
twilio
gradio>=5.0
gradio-webrtc
onnxruntime-gpu
We'll use the ONNX runtime to speed up YOLOv10 inference. This guide assumes you have access to a GPU. If you don't, change onnxruntime-gpu
to onnxruntime
. Without a GPU, the model will run slower, resulting in a laggy demo.
We'll use OpenCV for image manipulation and the Gradio WebRTC custom component to use WebRTC under the hood, achieving near-zero latency.
Note: If you want to deploy this app on any cloud provider, you'll need to use the free Twilio API for their TURN servers. Create a free account on Twilio. If you're not familiar with TURN servers, consult this guide.
We'll download the YOLOv10 model from the Hugging Face hub and instantiate a custom inference class to use this model.
The implementation of the inference class isn't covered in this guide, but you can find the source code here if you're interested. This implementation borrows heavily from this github repository.
We're using the yolov10-n
variant because it has the lowest latency. See the Performance section of the README in the YOLOv10 GitHub repository.
from huggingface_hub import hf_hub_download
from inference import YOLOv10
model_file = hf_hub_download(
repo_id="onnx-community/yolov10n", filename="onnx/model.onnx"
)
model = YOLOv10(model_file)
def detection(image, conf_threshold=0.3):
image = cv2.resize(image, (model.input_width, model.input_height))
new_image = model.detect_objects(image, conf_threshold)
return new_image
Our inference function, detection
, accepts a numpy array from the webcam and a desired confidence threshold. Object detection models like YOLO identify many objects and assign a confidence score to each. The lower the confidence, the higher the chance of a false positive. We'll let users adjust the confidence threshold.
The function returns a numpy array corresponding to the same input image with all detected objects in bounding boxes.
The Gradio demo is straightforward, but we'll implement a few specific features:
WebRTC
custom component to ensure input and output are sent to/from the server with WebRTC. time_limit
parameter of the stream
event. This parameter sets a processing time for each user's stream. In a multi-user setting, such as on Spaces, we'll stop processing the current user's stream after this period and move on to the next. We'll also apply custom CSS to center the webcam and slider on the page.
css = """.my-group {max-width: 600px !important; max-height: 600px !important;}
.my-column {display: flex !important; justify-content: center !important; align-items: center !important;}"""
with gr.Blocks(css=css) as demo:
gr.HTML(
"""
<h1 style='text-align: center'>
YOLOv10 Webcam Stream (Powered by WebRTC ⚡️)
</h1>
"""
)
with gr.Column(elem_classes=["my-column"]):
with gr.Group(elem_classes=["my-group"]):
image = WebRTC(label="Stream", rtc_configuration=rtc_configuration)
conf_threshold = gr.Slider(
label="Confidence Threshold",
minimum=0.0,
maximum=1.0,
step=0.05,
value=0.30,
)
image.stream(
fn=detection, inputs=[image, conf_threshold], outputs=[image], time_limit=10
)
if __name__ == "__main__":
demo.launch()
Our app is hosted on Hugging Face Spaces here.
You can use this app as a starting point to build real-time image applications with Gradio. Don't hesitate to open issues in the space or in the WebRTC component GitHub repo if you have any questions or encounter problems.