Configure online demo

这篇文章总共纪录了学习Replicate和HuggingFace在线搭demo的过程，最终我选的是HuggingFace，所以HuggingFace的内容详尽一些，Replicate只有一点点。

Replicate

参考：Replicate文档
Replicate是一个云端的机器学习模型运行平台，它允许用户使用云端API（在Python或Jupyter Notebook中）直接运行模型，并在云端进行模型的部署和调优。
其优势在于：
- 无需下载、安装或配置
- 快速轻松运行机器学习模型
- 提供大量的预训练模型和数据集

1.1 大致流程

Replicate的HTTP API 可与任何编程语言配合使用。使用 Python 客户端，首先需要安装Python库：pip install replicate。

然后去account settings中找到你的API token，并把它设置（声明）到你当前的代码环境：export REPLICATE_API_TOKEN=<paste-your-token-here>。

安全一点的方式为：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
# get a token: https://replicate.com/account
from getpass import getpass
import os

REPLICATE_API_TOKEN = getpass()
os.environ["REPLICATE_API_TOKEN"] = REPLICATE_API_TOKEN

## 2.1 在Python代码中导入Replicate库，以便使用Replicate的功能
import replicate
### receive images as inputs. Use a file handle or URL:
image = open("mystery.jpg", "rb")
### or...
image = "https://example.com/mystery.jpg"

## 2.2 调用模型，按需求返回
replicate.run(
  "replicate/resnet:dd782a3d531b61af491d1026434392e8afb40bfb53b8af35f727e80661489767",
  input={"image": image}
)

有个跟我很像的用了controlnet的工作在Replicate上的部署教程
大概了解了一下，感觉Replicate挺简单的，但是好像不如Huggingface的使用者多以及GPU算力要贵一点。决定临阵倒戈，还是去学HuggingFace吧，谁知道以后遇到什么bug了能不能解决。

好多人用啊，让我看看这个最大的开源平台有多牛(ง๑ •̀_•́)ง

Gradio + HuggingFace

计算机视觉和图像处理的算法一般都具有直观的实用性。为了推广自己工作的影响力，大家会选择把自己算法的实现效果部署到网页端的UI接口供大家使用。

1 界面设计库Gradio

Gradio是MIT的开源项目，它允许我们快速建立demo或者web application。使用时可理解为一个Python包，Prerequisite: Gradio requires Python 3.8 or higher，它的安装命令：pip install gradio。
Gradio教程

1.1 Quickstart

1.1.1 示例，实现一个本地静态交互页面

1
2
3
4
5
6
7
8
9
10
11
12
13
import gradio as gr

def greet(name, intensity):
    return "Hello, " + name + "!" * int(intensity)

demo = gr.Interface(
    fn=greet,
    inputs=["text", "slider"],
    outputs=["text"],
)

demo.launch()

我用的anaconda命令行运行，它显示：

1
2
3
Running on local URL:  http://127.0.0.1:7860

To create a public link, set `share=True` in `launch()`.

然后我在本地浏览器输入http://127.0.0.1:7788/就看到了对应的页面。

1.1.2 hot reload mode

Automatically reloads the Gradio app whenever you make changes to the file. To do this, simply type in gradio before the name of the file instead of python. In the example:gradio app.py(Type this in terminal).
Hot Reloading Guide

1.1.3 Understanding the Interface Class

gradio的核心是它的gr.Interface函数，用来构建可视化界面。The Interface class is designed to create demos for machine learning models which accept one or more inputs, and return one or more outputs.
它主要有三个核心属性：

  • fn: the function to wrap a user interface (UI) around
  • inputs：the number of components should match the number of arguments in your function
  • outputs：the number of components should match the number of return values from your function.

所以对于任何图像处理类的ML代码来说，基本流程就是图像输入>>模型推理>>返回图片。building Interfaces

1.1.4 Sharing Your Demo

Gradio lets you easily share a machine learning demo without having to worry about the hassle of hosting on a web server. Simply set share=True in launch() like demo.launch(share=True), and a publicly accessible URL will be created for your demo.

所以实际上他跑起来用的是我本地的算力资源，但是类似网络通信可以将输入输出在不同电脑之间进行联络。Share links expire after 72 hours. (it is also possible to set up your own Share Server on your own cloud server to overcome this restriction.)

1.2 the Interface Class

1.2.1 Components

提供了超过30种components(e.g. the gr.Image component is designed to handle input or output images, the gr.Label component displays classification labels and probabilities, the gr.Plot component displays various kinds of plots, and so on).
而且Gradio可以自动处理输入输出与函数fn之间的类型转换(Preprocessing and Postprocessing)，需要在对应的控件里加一个属性type：img = gr.Image(type="pil")

1.2.2 Components Attributes

比如加一个滑轮效果

1
2
3
4
5
6
7
8
9
10
11
12
13
14
# 之前的
demo = gr.Interface(
    fn=greet,
    inputs=["text", "slider"],
    outputs=["text"],
)
######################################################################
# 现在
demo = gr.Interface(
    fn=greet,
    inputs=["text", gr.Slider(value=2, minimum=1, maximum=10, step=1)],
    outputs=[gr.Textbox(label="greeting", lines=3)],
)

Suppose you had a more complex function, with multiple outputs as well. In the example below, we define a function that takes a string, boolean, and number, and returns a string and number.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
# 之前的
def greet(name, intensity):
    return "Hello, " + name + "!" * intensity

demo = gr.Interface(
    fn=greet,
    inputs=["text", gr.Slider(value=2, minimum=1, maximum=10, step=1)],
    outputs=[gr.Textbox(label="greeting", lines=3)],
)
# lines=3 参数用于设置文本框的默认显示行数
# 具体来说，它控制了文本框的高度，使其在显示时能够容纳 3 行文本。

######################################################################
# 现在
def greet(name, is_morning, temperature):
    salutation = "Good morning" if is_morning else "Good evening"
    greeting = f"{salutation} {name}. It is {temperature} degrees today"
    celsius = (temperature - 32) * 5 / 9
    return greeting, round(celsius, 2)

demo = gr.Interface(
    fn=greet,
    inputs=["text", "checkbox", gr.Slider(0, 100)],
    outputs=["text", "number"],
)

1.2.3 Reactive Interfaces

提供Live Interfaces模式，不需要submit button，会自动计算提交结果。暂时我不太需要这个功能，之后遇到再学。

1.2.4 The 4 Kinds of Gradio Interfaces

  1. Standard demos: which have both separate inputs and outputs
  2. Output-only demos: which don’t take any input but produce on output
  3. Input-only demos: which don’t produce any output but do take in some sort of input
  4. Unified demos: which have both input and output components, but the input and output components are the same. This means that the output produced overrides the input (e.g. a text autocomplete model)

1.2.5 Queuing

Every Gradio app comes with a built-in queuing system that can scale to thousands of concurrent users. You can configure the queue by using queue() method. 例如通过设置queue()的default_concurrency_limit参数来控制单次处理的请求数(默认是1):

1
2
demo = gr.Interface(...).queue(default_concurrency_limit=5)
demo.launch()

1.2.6 Streaming outputs

In some cases, you may want to stream a sequence of outputs rather than show a single output at once.官网说的场景是“图像生成，显示到结果图像中的每一时刻图像”，感觉我暂时不需要，之后再学。

1.2.7 Alerts

跟用户显示警告提醒他们某些输入输出的错误。主要使用gr.Error("custom message")、gr.Warning("custom message")和gr.Info("custom message") 。The only difference between gr.Info() and gr.Warning() is the color of the alert.

1.2.8 Styling

关于Theming的介绍
可以直接调用别人目前设计的主题模板，也可以自己上传CSS文件自定义。Gradio 具有内置主题引擎，可自定义应用的外观和感觉。只需要在代码中写上这一句demo = gr.Interface(..., theme=gr.themes.Monochrome())。
现有的一系列主题可以通过gr.themes.*进行调用，有：

1
2
3
4
5
gr.themes.Base()
gr.themes.Default()
gr.themes.Glass()
gr.themes.Monochrome()
gr.themes.Soft()

1.2.8.1 Using the Theme Builder

然后基于这些现有的模板，你可以任意调整其中的一些类的属性

1
2
3
4
5
6
7
8
9
10
11
12
## 原先
import gradio as gr
gr.themes.builder()

## 修改后
import gradio as gr
theme = gr.themes.Base().set(
    button_secondary_text_color_dark='*secondary_900'
)

with gr.Blocks(theme=theme) as demo:
    ...

1.2.8.1 Extending Themes via the Constructor

Although each theme has hundreds of CSS variables, the values for most these variables are drawn from 8 core variables which can be set through the constructor of each prebuilt theme. Modifying these 8 arguments allows you to quickly change the look and feel of your app.

  • ① Core Colors：gradio.themes.Color
    • these Color objects hold brightness values for the palette of a single hue, ranging from 50, 100, 200…, 800, 900, 950.
    • primary_hue: This is the color draws attention in your theme. In the default theme, this is set to gradio.themes.colors.orange.
    • secondary_hue: This is the color that is used for secondary elements in your theme. In the default theme, this is set to gradio.themes.colors.blue.
    • neutral_hue: This is the color that is used for text and other neutral elements in your theme. In the default theme, this is set to gradio.themes.colors.gray.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
with gr.Blocks(theme=gr.themes.Default(primary_hue="red", secondary_hue="pink")) as demo:

## 或者
with gr.Blocks(theme=gr.themes.Default(primary_hue=gr.themes.colors.red, secondary_hue=gr.themes.colors.pink)) as demo:
    ...

## 或者modify the values of CSS variables after the theme has been loaded
theme = gr.themes.Default(primary_hue="blue").set(
    loader_color="#FF0000",
    slider_color="#FF0000",
)

with gr.Blocks(theme=theme) as demo:
    ...

## 支持的颜色字符有
slate
gray
zinc
neutral
stone
red
orange
amber
yellow
lime
green
emerald
teal
cyan
sky
blue
indigo
violet
purple
fuchsia
pink
rose

  • ② Core Sizing：gradio.themes.Size
    • These Size objects hold pixel size values that range from xxs to xxl.
    • spacing_size: gradio.themes.sizes.spacing_md.
    • radius_size: gradio.themes.sizes.radius_md.
    • text_size: gradio.themes.sizes.text_md.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
with gr.Blocks(theme=gr.themes.Default(spacing_size="sm", radius_size="none")) as demo:
    ...
with gr.Blocks(theme=gr.themes.Default(spacing_size=gr.themes.sizes.spacing_sm, radius_size=gr.themes.sizes.radius_none)) as demo:
    ...

## 支持的size object有
radius_none
radius_sm
radius_md
radius_lg
spacing_sm
spacing_md
spacing_lg
text_sm
text_md
text_lg
  • ③ Core Fonts：gradio.themes.xxxFont
    • set the fonts of the theme.
    • font: gradio.themes.GoogleFont("Source Sans Pro").
    • font_mono: sets the monospace font of the theme gradio.themes.GoogleFont("IBM Plex Mono").
1
2
3
4
5
6
7
8
9
10
11
12
13
with gr.Blocks(theme=gr.themes.Default(font=[gr.themes.GoogleFont("Inconsolata"), "Arial", "sans-serif"])) as demo:
    ...
## 支持的size object有
radius_none
radius_sm
radius_md
radius_lg
spacing_sm
spacing_md
spacing_lg
text_sm
text_md
text_lg

1.2.9 Progress bars

挺人性化的设计的，到哪都是进度条是吧( ´◔︎ ‸◔︎`)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
import gradio as gr
import time

def slowly_reverse(word, progress=gr.Progress()):
    progress(0, desc="Starting")
    time.sleep(1)
    progress(0.05)
    new_string = ""
    for letter in progress.tqdm(word, desc="Reversing"):
        time.sleep(0.25)
        new_string = letter + new_string
    return new_string

demo = gr.Interface(slowly_reverse, gr.Text(), gr.Text())

demo.launch()

you can even report progress updates automatically from any tqdm.tqdm that already exists within your function by setting the default argument as gr.Progress(track_tqdm=True)!

TODO

add example
我们可以在页面下方添加供用户选择的测试样例。比如做了一个图像去噪算法，但是用户手头并没有躁点照片，example能让他更快的体验到效果

1
2
3
4
5
6
7
8
9
10
11
12
13
import gradio as gr

def greet(name, intensity):
    return "Hello, " + name + "!" * int(intensity)

demo = gr.Interface(
    fn=greet,
    inputs=["text", "slider"],
    outputs=["text"],
)

demo.launch()

2 在线托管平台 HuggingFace

前文提到过通过在代码中设置demo.launch(share=True)就能生成一个共享链接形如：https://07ff8706ab.gradio.live。虽然该链接是通过 Gradio 共享服务器提供的免费公共 URL，但实际上跑代码的算力是我本地的服务器，同时共享链接将在 72 小时后过期。所以接下来的问题就是怎么在自己的云服务器上设置自己的共享服务器，这样才能突破链接的时间限制。而这就是HuggingFace可以干的事！Hugging Face Spaces 提供基础设施，进而可以免费永久托管机器学习模型！
使用服务前，先注册账号（官网界面右上角”Sign Up”）

2.1 Hugging Face Spaces

Spaces文档
在HuggingFace官网登录账号后，切换到Spaces创建一个新的空间假设叫“DemoName”。先选”Gradio”和”Private”，以生成一个可使用的Gradio在线应用：https://huggingface.co/spaces/YourUserName/DemoName。每个项目空间默认免费配备8个CPU核和16GB 运行内存，GPU资源需要单独付费。Each Spaces environment is limited to 16GB RAM, 2 CPU cores and 50GB of (not persistent) disk space by default.感觉这个比github还适合进行代码备份

创建完Space之后，把本地预先构建好的项目文件（UI构建文件必须得命名成app.py且位于根目录）上传到该空间。具体方法与Github项目的上传和版本维护方式完全一样:

  • 克隆space项目到本地：git clone https://huggingface.co/spaces/your_account/proj_name/tree/main
  • 将本地已跑通的项目文件复制到刚才克隆的space项目文件夹
  • 新建描述运行环境依赖的文件：requirements.txt
  • 指定Python依赖的包:packages.txt
  • 指定特殊的系统依赖配置
  • 详情参考
  • 将此更新同步到远程仓库：
    • git add -A .
    • git commit -m “add project files”
    • git push

完成以上步骤后（等待1~2分钟系统刷新），进入Space项目的App选项卡即可查看部署到web端的应用。

Tutorial > Tools
#Tools
Configure online demo
http://example.com/2024/05/19/6_onlineDemo/
Author
Xi Wang
Posted on
May 19, 2024
Licensed under