教程集 www.jiaochengji.com
教程集 >  Golang编程  >  正文 golang物联网_使用golang谷歌云平台和grafana监控物联网设备

golang物联网_使用golang谷歌云平台和grafana监控物联网设备

发布时间:2021-12-10   编辑:jiaochengji.com
教程集为您提供golang物联网,使用golang谷歌云平台和grafana监控物联网设备等资源,欢迎您收藏本站,我们将为您提供最新的golang物联网,使用golang谷歌云平台和grafana监控物联网设备资源
<article style="font-size: 16px;">

golang物联网

<section>

In this article, we’ll be setting up step by step, a monitoring environment for IoT devices with an Arduino based board (ESP32), Grafana, and Google Cloud Platform tools. We’ll be using other tools like Docker and Prometheus, you don’t need to know how to use them, but I highly recommend that you know what they are and what they do.

在本文中,我们将逐步设置一个使用基于Arduino的板( ESP32 ), GrafanaGoogle Cloud Platform工具对IoT设备进行监视的环境。 我们将使用其他工具,例如Docker Prometheus ,您不需要知道如何使用它们,但是我强烈建议您了解它们的用途。

First, let’s take a look into our infrastructure:

首先,让我们看一下我们的基础架构:

<figure style="display:block;text-align:center;">
</figure>

We’re using the built-in temperature sensor from ESP32 to sent data every 5 seconds to a pub/sub service in GCP. This data will be processed by a Go service and sent to Prometheus, and we’re using Grafana to visualize this data.

我们使用ESP32内置的温度传感器每5秒将数据发送到GCP中发布/订阅服务。 这些数据将由Go服务处理并发送到Prometheus ,我们正在使用Grafana可视化此数据。

Grafana is a visualization tool. Prometheus is a data source for Grafana that collects the data in time series and displays in a way that Grafana understands. Since Prometheus can’t collect data directly to pub/sub, we need a third service to send it to Prometheus.

Grafana是一种可视化工具。 Prometheus是Grafana的数据源,它按时间序列收集数据并以Grafana理解的方式显示。 由于Prometheus无法直接将数据收集到pub / sub,因此我们需要第三项服务将其发送给Prometheus。

I made a Github repository with all the codes used and instructions in this article, fell free to check it out.

我创建了一个Github存储库,其中包含本文中使用的所有代码和说明,可以随意查看。

So let’s get our hands dirty!

因此,让我们动手吧!

<h1> Google Cloud Platform入门 <span style="font-weight: bold;">(</span>Getting started with Google Cloud Platform<span style="font-weight: bold;">)</span></h1>

On Google Cloud, we’ll be using Core IoT to manage our devices, pub/sub as messaging system and Google Run to host our containers.

在Google Cloud上,我们将使用Core IoT管理设备,将发布/订阅作为消息传递系统,并在Google Run中托管容器。

First, let’s set up our project. You’ll need a Google account and a credit card, but don’t worry you won’t be charged for anything (if you don’t do some heavy work), your free trial lasts for 3 months and you have US$300 to spend in any Google Cloud service. But you can always keep an eye on your billing board to not have any surprises on your credit card.

首先,让我们建立项目。 您需要一个Google帐户和一张信用卡,但是不用担心,您无需支付任何费用(如果您不做繁重的工作),免费试用期为3个月,您有300美元在任何Google Cloud服务中花费。 但是,您始终可以随时注意帐单上的内容,以确保信用卡不会出现任何意外。

I’ll use the Google Cloud CLI to set up our environment but feel free to use the web console. Install it for your operational system.

我将使用Google Cloud CLI设置环境,但可以随时使用Web控制台。 为您的操作系统安装它。

To make things easier, you can export this environment variables and just paste the commands from this tutorial (choose your own names):

为了简化操作,您可以导出此环境变量,然后仅粘贴本教程中的命令(选择您自己的名称):

<pre><code class="has">export PROJECT_ID=export REGION=export TOPIC_ID=export SUBSCRIPTION=export REGISTRY=export DEVICE_ID=export USER_NAME=export IMAGE_NAME=export SERVICE_NAME=</code></pre> <blockquote>

The export command should look like this:

导出命令应如下所示:

<code><em>export PROJECT_ID=temperature-grafana</em></code>

<code><em>export PROJECT_ID=temperature-grafana</em></code>

</blockquote>

To start, log in with your Google account on CLI, create a project, and select the project created. Open a terminal and type the following commands:

首先,在CLI上使用您的Google帐户登录,创建一个项目,然后选择创建的项目。 打开终端并输入以下命令:

<pre><code class="has">$ gcloud auth login
$ gcloud projects create $PROJECT_ID
$ gcloud config set project $PROJECT_ID</code></pre>

You can check your project dashboard, and if everything goes well, you should see your project there.

您可以检查项目仪表板,如果一切顺利,则应该在此处看到您的项目。

<figure style="display:block;text-align:center;">
</figure>

Now let's enable pub/sub and IoT Core services in our project. But before that, you'll need to enable the billing into your project. To do that, run the following command and continue to the browser and link a profile:

现在,让我们在项目中启用发布/订阅IoT核心服务。 但是在此之前,您需要在您的项目中启用计费功能。 为此,请运行以下命令并继续浏览器并链接配置文件:

<pre><code class="has">$ open "https://console.cloud.google.com/billing/linkedaccount?project=$PROJECT_ID"
$ gcloud services enable cloudiot.googleapis.com pubsub.googleapis.com</code></pre>

We also need to permit IoT Core to publish into pub/sub service. Since IoT Core is responsible for our devices, and they don’t need to subscribe to any topic, we’re giving them just the publishing role.

我们还需要允许IoT Core发布到发布/订阅服务中。 由于IoT核心负责我们的设备,并且它们不需要订阅任何主题,因此我们只赋予他们发布角色。

<pre><code class="has">$ gcloud projects add-iam-policy-binding $PROJECT_ID \
--member=serviceAccount:cloud-iot@system.gserviceaccount.com \
--role=roles/pubsub.publisher</code></pre>

Choose a region here. I’m using us-central1, but pick the one that better suits you. We also need a pub/sub topic, a subscription, and a registry.

在这里选择一个地区。 我正在使用us-central1 ,但请选择一个更适合您的。 我们还需要一个pub / sub主题,一个subscription和一个注册表。

On MQTT, pub/sub works like an Instagram/Twitter hashtag, where you can publish a post using a hashtag and who is following (or subscribed) to that hashtag, we'll see your post. The same works for MQTT, but the hashtag is the topic, the photo is the message and the people following that topic is the subscription.

在MQTT上,发布/订阅的工作方式类似于Instagram / Twitter的标签,您可以在其中使用标签来发布帖子,并且关注(或订阅)该标签的人,我们将看到您的帖子。 MQTT的工作原理相同,但主题标签是主题标签,照片是消息,主题之后的人员是订阅。

A registry is like a bucket for our IoT devices. It allows us to group devices and set properties that they all share, such as connection protocol, data storage location, and Cloud pub/sub topics.

注册表就像我们的物联网设备的存储桶。 它使我们可以对设备进行分组并设置它们都共享的属性,例如连接协议,数据存储位置和Cloud pub / sub主题。

Follow these commands:

请遵循以下命令:

<pre><code class="has">$ gcloud pubsub topics create $TOPIC_ID
$ gcloud pubsub subscriptions create --topic $TOPIC_ID $SUBSCRIPTION
$ gcloud iot registries create $REGISTRY \
--region=$REGION \
--event-notification-config=topic=temperature-topic \
--enable-mqtt-config --enable-http-config</code></pre>

You can check your registries, and if everything goes well, you should see your registry with your topic and subscription there.

您可以检查注册表,如果一切顺利,则应该在其中看到带有主题和订阅的注册表。

<figure style="display:block;text-align:center;">
</figure><h1> 设置ESP32 <span style="font-weight: bold;">(</span>Setting up ESP32<span style="font-weight: bold;">)</span></h1>

We’ll be using the Espressif micro-controller ESP32 for its WiFi and a built-in temperature sensor. Also, I’m using the Arduino IDE, so make sure you have it installed and set up for ESP32 usage, if you need some help you can follow this tutorial:

我们将使用Espressif微控制器ESP32的WiFi和内置温度传感器。 另外,我使用的是Arduino IDE,因此请确保已安装并设置为ESP32使用,如果需要帮助,可以按照本教程进行操作:

We need to generate an Elliptic Curve (EC) ES256 private/public key pair for our device authentication. Make sure to generate them into a “safe place”:

我们需要为设备认证生成一个椭圆曲线(EC)ES256私钥/公钥对。 确保将它们生成到“安全的地方”:

<pre><code class="has">$ openssl ecparam -genkey -name prime256v1 -noout -out ec_private.pem$ openssl ec -in ec_private.pem -pubout -out ec_public.pem</code></pre>

Now we have to register our device into Core IoT, so run the following commands:

现在我们必须将设备注册到Core IoT中,因此运行以下命令:

<pre><code class="has">$ gcloud iot devices create $DEVICE_ID \
--region=$REGION \
--registry=$REGISTRY \
--public-key="path=./ec_public.pem,type=es256"</code></pre>

Install “Google Cloud IoT Core JWT” and lwmMQTT from Joel Garhwller libraries on your Arduino IDE. They’re responsible for connecting, authenticating, and sending messages to GCP.

从Arduino IDE上的Joel Garhwller库安装“ Google Cloud IoT Core JWT”和lwmMQTT。 他们负责连接,验证消息并将消息发送到GCP。

<figure style="display:block;text-align:center;">
</figure>

Now let's use the library code example for ESP32-lwmqtt:

现在使用ESP32-lwmqtt的库代码示例:

<figure style="display:block;text-align:center;">
</figure>

On ciotc_config.h, set your WiFi network and credentials:

ciotc_config.h上,设置您的WiFi网络和凭据:

<pre><code class="has">// Wifi network details
const char *ssid = "SSID";
const char *password = "PASSWORD";// Cloud iot details
const char *project_id = "project-id";
const char *location = "us-central1";
const char *registry_id = "my-registry";
const char *device_id = "my-esp32-device";</code></pre>

To get your private_key_str, run the following command at the same directory where you saved your public/private keys and paste the "priv" it into the code:

要获取您的private_key_str ,请在保存公钥/私钥的同一目录中运行以下命令,并将“ priv ”粘贴到代码中:

<pre><code class="has">$ openssl ec -in ec_private.pem -noout -text</code></pre>

<em>PS: The key length should be 32 pairs of hex digits. If your private key is bigger, remove the “00:” and if its smaller add “00:”. It should look like this:</em>

<em>PS:密钥长度应为32对十六进制数字。</em> <em>如果私钥较大,则删除“ 00:”,如果较小,则添加“ 00:”。</em> <em>它看起来应该像这样:</em>

<figure style="display:block;text-align:center;">
</figure>

You’ll need to set up your root_cert as well. Do the same steps as previously:

您还需要设置root_cert 。 执行与之前相同的步骤:

<pre><code class="has">$ openssl s_client -showcerts -connect mqtt.googleapis.com:8883</code></pre>

It should look something like this:

它看起来应该像这样:

<figure style="display:block;text-align:center;">
<figcaption>Dracula Theme from Zeno Rocha. Dracula Theme 。 </figcaption></figure>

On Esp32-lwmqtt.ino file, let's do some changes to get the ESP32 temperature. This is how our code looks like:

Esp32-lwmqtt.ino文件上,进行一些更改以获取ESP32温度。 我们的代码如下所示:

<pre class="has"><code>#include "esp32-mqtt.h" #ifdef __cplusplus extern "C" { #endif uint8_t temprature_sens_read(); #ifdef __cplusplus } #endif uint8_t temprature_sens_read(); void setup() { Serial.begin(115200); pinMode(LED_BUILTIN, OUTPUT); setupCloudIoT(); } unsigned long lastMillis = 0; void loop() { mqtt->loop(); delay(10); if (!mqttClient->connected()) { connect(); } if (millis() - lastMillis > 5000) { lastMillis = millis(); const float temperature = (temprature_sens_read() - 32) / 1.8; String payload = String("{\"temperature\":") String(temperature) String("}"); publishTelemetry(payload); } }</code></pre>

Now plug in your ESP32 into the USB and upload the sketch. Open the console, set it to 115200 baud rate and the logs should look like this:

现在将ESP32插入USB并上传草图。 打开控制台,将其设置为115200波特率,日志应如下所示:

<figure style="display:block;text-align:center;">
</figure>

Now if you check your pub/sub topics, you may see something like this:

现在,如果您查看pub / sub主题,您可能会看到类似以下内容:

<figure style="display:block;text-align:center;">
</figure><h1> Golang指标转换器 <span style="font-weight: bold;">(</span>Golang Metrics Converter<span style="font-weight: bold;">)</span></h1>

We need someone subscribed to the topic where our device is publishing and send those messages to Prometheus. So let’s create a service with Golang and put it into a container. First, we need a service-account and give it the pub/sub subscribe role:

我们需要有人订阅我们设备发布的主题,然后将这些消息发送给Prometheus。 因此,让我们使用Golang创建服务并将其放入容器中。 首先,我们需要一个服务帐户并将其授予pub / sub订阅角色:

<pre><code class="has">$ gcloud iam service-accounts create $USER_NAME$ gcloud projects add-iam-policy-binding $PROJECT_ID \
--member=serviceAccount:$USER_NAME@$PROJECT_ID.iam.gserviceaccount.com \
--role=roles/pubsub.subscriber</code></pre>

There are several ways to our service authenticate into GCP, so let’s use the JSON file method. So let’s generate a file with credentials for the service account we just created and export its path to an environment variable called GOOGLE_APPLICATION_CREDENTIALS:

我们的服务可以通过多种方式向GCP进行身份验证,因此让我们使用JSON文件方法。 因此,让我们为刚刚创建的服务帐户生成一个带有凭证的文件,并将其路径导出到名为GOOGLE_APPLICATION_CREDENTIALS的环境变量中

<pre><code class="has">$ gcloud iam service-accounts keys create resources/service-account-key.json --iam-account $USER_NAME@$PROJECT_ID.iam.gserviceaccount.com

$ export GOOGLE_APPLICATION_CREDENTIALS=$CREDENTIAL_PATH</code></pre> <blockquote>

The export command should look like this:

导出命令应如下所示:

<code>export GOOGLE_APPLICATION_CREDENTIALS=/home/leo/resources/service-account-key.json</code>

<code>export GOOGLE_APPLICATION_CREDENTIALS=/home/leo/resources/service-account-key.json</code>

</blockquote>

Now let's create our Golang service:

现在让我们创建我们的Golang服务:

<pre class="has"><code>package main import ( "context" "encoding/json" "log" "net/http" "strings" "sync" "cloud.google.com/go/pubsub" "github.com/prometheus/client_golang/prometheus" "github.com/prometheus/client_golang/prometheus/promauto" "github.com/prometheus/client_golang/prometheus/promhttp" ) // Payload receveid from pub/sub type Payload struct { Temperature float64 `json:"temperature"` } // Gauge metric temperature for Prometheus var ( temperatureGauge = promauto.NewGauge(prometheus.GaugeOpts{ Name: "temperature_gauge", Help: "Temperature in °C", }) ) // Pulling messages from pub/sub func pullMessages() { log.Println("STARTED PULLING MESSAGES") ctx := context.Background() // Set your $PROJECT_ID client, err := pubsub.NewClient(ctx, "temperature-grafana") if err != nil { log.Fatal(err) } // Set your $SUBSCRIPTION subID := "temperature-subscription" var mu sync.Mutex sub := client.Subscription(subID) cctx, cancel := context.WithCancel(ctx) err = sub.Receive(cctx, func(ctx context.Context, msg *pubsub.Message) { mu.Lock() defer mu.Unlock() log.Print("Got message: " string(msg.Data)) if !containsInitPayload(string(msg.Data)) { var t Payload err := json.Unmarshal(msg.Data, &t) if err != nil { panic(err) } temperatureGauge.Set(t.Temperature) } msg.Ack() }) if err != nil { cancel() log.Fatal(err) } cancel() } func containsInitPayload(payload string) bool { if strings.Contains(payload, "esp32-connected") { return true } return false } func main() { go pullMessages() log.Println("STARTED PROMETHEUS") http.Handle("/metrics", promhttp.Handler()) http.ListenAndServe(":2112", nil) }</code></pre>

And a Dockerfile for it:

还有一个Dockerfile:

<pre class="has"><code>FROM golang COPY . /app WORKDIR /app ENV GOOGLE_APPLICATION_CREDENTIALS=/app/resources/service-account-key.json RUN go mod download CMD ["go", "run", "pubsub.go"]</code></pre> <h1>Google Cloud Run<span style="font-weight: bold;">(</span>Google Cloud Run<span style="font-weight: bold;">)</span></h1>

Cloud Run it's an awesome fully managed compute platform for deploying and scaling containerized applications quickly and securely. If you’re new to it, check this video

Cloud Run是一个很棒的完全托管的计算平台,可快速安全地部署和扩展容器化的应用程序。 如果您是新手,请观看此视频

<figure style="display:block;text-align:center;">

演示地址

</figure>

First, we need to enable Cloud Build, Cloud Run, and Container Registry in our project:

首先,我们需要在我们的项目中启用Cloud Build,Cloud Run和Container Registry:

<pre><code class="has">$ gcloud services enable cloudbuild.googleapis.com run.googleapis.com containerregistry.googleapis.com</code></pre>

Now let’s build and push our Golang service Docker image to the Cloud Build:

现在,让我们构建并将Golang服务Docker映像推送到Cloud Build:

<pre><code class="has">$ gcloud builds submit --tag gcr.io/$PROJECT_ID/$IMAGE_NAME</code></pre>

To deploy it, run this command:

要部署它,请运行以下命令:

<pre><code class="has">$ gcloud run deploy $SERVICE_NAME --image gcr.io/$PROJECT_ID/ $IMAGE_NAME --region us-central1 --plataform managed --allow-authenticated --port 2112</code></pre>

GCP will generate an URL for your container, copy it. You can get it on your terminal or accessing your project on your project Cloud Run page.

GCP将为您的容器生成一个URL,然后将其复制。 您可以在终端上获取它,也可以在项目“云运行”页面上访问项目。

<figure style="display:block;text-align:center;">
</figure>

Now paste it on your prometheus.yml file, on your targets (remove the https://):

现在将其粘贴到您的目标的prometheus.yml文件中(删除https://):

<pre class="has"><code>global: scrape_interval: 10s evaluation_interval: 10s external_labels: monitor: 'codelab-monitor' scrape_configs: - job_name: 'temperature' scrape_interval: 5s static_configs: - targets: - 'temperature-grafana-utsma6q3sq-uc.a.run.app' # Your project URL</code></pre>

Since we can't deploy existing images from Docker Hub to Cloud Run, we need to make a custom Docker image for Prometheus and Grafana, then deploy them. Let's deploy a Prometheus container:

由于我们无法将现有映像从Docker Hub部署到Cloud Run,因此我们需要为Prometheus和Grafana创建自定义Docker映像,然后进行部署。 让我们部署一个Prometheus容器:

<pre class="has"><code>FROM prom/prometheus ADD ./prometheus.yml /etc/prometheus/prometheus.yml EXPOSE 9090</code></pre>

Building and submitting to production:

建立并提交生产:

<pre><code class="has">$ gcloud builds submit --tag gcr.io/$PROJECT_ID/prometheus .
$ gcloud run deploy prometheus --image gcr.io/$PROJECT_ID/prometheus --region $REGION --platform managed --allow-unauthenticated --port 9090</code></pre>

Make sure to save the URL generated. Same thing for Grafana:

确保保存生成的URL。 Grafana也是这样:

<pre class="has"><code>FROM grafana/grafana EXPOSE 3000 ENTRYPOINT [ "/run.sh" ]</code></pre>

Building and submitting to production:

建立并提交生产:

<pre><code class="has">$ gcloud builds submit --tag gcr.io/$PROJECT_ID/grafana .$ gcloud run deploy grafana --image gcr.io/$PROJECT_ID/grafana --region $REGION --platform managed --allow-unauthenticated --port 3000</code></pre>

Now you can access your Grafana board through the generated URL. You can log in with the admin login (default user: admin, pass: admin, make sure to change that).

现在,您可以通过生成的URL访问Grafana板。 您可以使用admin登录名登录(默认用户:admin,pass:admin ,请确保进行更改)。

<figure style="display:block;text-align:center;">
</figure>

Now we have to set up Grafana to listen to our Prometheus. After logging in, go to "Data Source" on the right menu bar, click on "Add data source" and select Prometheus.

现在,我们必须设置Grafana来收听我们的Prometheus。 在登录后,进入“数据源”右侧的菜单栏上,单击“添加数据源”,然后选择普罗米修斯

<figure style="display:block;text-align:center;">
</figure>

On the Prometheus data source page, paste the URL to your Prometheus instance on the HTTP > URL and hit "save & test".

在Prometheus数据源页面上,将该URL粘贴到HTTP> URL上的Prometheus实例中,然后单击“保存并测试”。

<figure style="display:block;text-align:center;">
</figure>

On my Github repository, there’s a JSON file that will import a Grafana Dashboard. Feel free to use it or create your own. If you looking into creating your dashboards, Grafana uses PromQL for querying metrics data, take a look into its documentation for more information.

在我的Github存储库中,有一个JSON文件将导入Grafana仪表板。 随意使用它或创建自己的。 如果您想创建仪表板,则Grafana使用PromQL查询指标数据,请查看其文档以获取更多信息。

To import my dashboard, go to the right menu bar, then to Create and Import. Paste the JSON content into the text box and hit load.

要导入我的仪表板,请转到右侧的菜单栏,然后​​转到“创建并导入”。 将JSON内容粘贴到文本框中,然后点击load

<figure style="display:block;text-align:center;">
</figure>

Select Prometheus as your data source and boom, you should a dashboard like this one:

选择Prometheus作为您的数据源和繁荣,您应该像这样一个仪表板:

<figure style="display:block;text-align:center;">
</figure>

Now plug your ESP32 on the USB and you should see the graph going up and down!

现在将ESP32插入USB,您应该会看到图形在上升和下降!

<figure style="display:block;text-align:center;">
</figure>

And that’s it. You can monitor data from IoT devices anywhere in the world. If you have some trouble setting this up, feel free to contact me on my email or connect with me on my LinkedIn and Github.

就是这样。 您可以监控来自世界各地的IoT设备的数据。 如果您在设置时遇到问题,请随时通过电子邮件与我联系,或者通过LinkedIn和Github与我联系。

</section>
<blockquote>

翻译自: https://medium.com/@leonardoaugusto287/monitoring-iot-devices-with-golang-google-cloud-platform-and-grafana-eb99cc36db4b

</blockquote>

golang物联网

</article> 到此这篇关于“golang物联网_使用golang谷歌云平台和grafana监控物联网设备”的文章就介绍到这了,更多文章或继续浏览下面的相关文章,希望大家以后多多支持JQ教程网!

您可能感兴趣的文章:
golang物联网_使用golang谷歌云平台和grafana监控物联网设备
Golang 微服务教程(一)
为互联网IT人打造的中文版awesome-go
为什么要学 Go
Go语言入门教程-介绍和安装
Deno 来势汹汹?这些最新项目让你知道Node.js可不是吃素的
golang微服务框架对比_最强开源微服务框架,全网独家整理
golang微服务框架对比_Golang 微服务教程(一)
学习golang开始前的准备工作
快速部署 Spring PetClinic 到函数计算平台

上一篇:Go语言基础(一) 下一篇:Golang实现KMP算法
[关闭]
~ ~