{
  "version": "https://jsonfeed.org/version/1", 
  "title": "Big Data", 
  "description": null, 
  "home_page_url": "https://www.v2ex.com/go/bigdata", 
  "feed_url": "https://www.v2ex.com/feed/bigdata.json", 
  "items": [
    {
      "author": {
        "url": "https://www.v2ex.com/member/louhubiao", 
        "name": "louhubiao", 
        "avatar": "https://cdn.v2ex.com/gravatar/74b9b3ffc3a1bead85e0e18c07b47ba1?s=73&d=retro"
      }, 
      "url": "https://www.v2ex.com/t/1107283", 
      "date_modified": "2025-01-23T03:13:26+00:00", 
      "content_html": "<p>\u4e4b\u524d\u7528 datax \u914d\u7f6e\uff0cmysql \u540c\u6b65\u5230 hdfs \u8fd8\u662f\u5f88\u597d\u7528\u7684\uff0c\u4f46\u662f\u9700\u8981\u624b\u5199\u914d\u7f6e</p>\n<p>\u6709\u6ca1\u6709\u66f4\u52a0\u6613\u7528\u7684\uff0c\u6bd4\u5982\u672c\u5730\u8d77\u4e2a\u955c\u50cf\u5c31\u884c\uff0c\u4ee5\u53ca\u53ef\u89c6\u5316\u914d\u7f6e\u8868\u540c\u6b65\uff0c\u914d\u7f6e\u540c\u6b65\u54ea\u4e9b\u5b57\u6bb5</p>\n<p>\u6700\u597d\u662f\u80fd\u548c\u5176\u4ed6\u7cfb\u7edf\u8fdb\u884c\u4ea4\u4e92\uff0c\u4f8b\u5982\u8c03\u5ea6\u7cfb\u7edf\uff0c\u5728\u6570\u636e\u540c\u6b65\u5b8c\u6210\u540e\u518d\u6267\u884c\u540e\u7eed\u4efb\u52a1</p>\n", 
      "date_published": "2025-01-23T03:12:47+00:00", 
      "title": "\u5982\u4f55\u5c06\u8fdc\u7aef mysql \u53d8\u66f4\u540c\u6b65\u5230\u672c\u5730", 
      "id": "https://www.v2ex.com/t/1107283"
    }, 
    {
      "author": {
        "url": "https://www.v2ex.com/member/rockagen", 
        "name": "rockagen", 
        "avatar": "https://cdn.v2ex.com/gravatar/3fcdce3929a21588a7489b3787b8ae0d?s=73&d=retro"
      }, 
      "url": "https://www.v2ex.com/t/932299", 
      "title": "\u5410\u69fd\u4e00\u4e0b\u56fd\u5185\u5f00\u6e90\u7684\u4e00\u4e9b\u4e1c\u897f", 
      "id": "https://www.v2ex.com/t/932299", 
      "date_published": "2023-04-13T09:26:58+00:00", 
      "content_html": "<p>\u7531\u4e8e\u5de5\u4f5c\u539f\u56e0\u9700\u8981\u540c\u6b65\u4e00\u4e9b\u6570\u636e\u5230 hive \uff0c\u5f53\u65f6\u9009\u578b\u6709 datax \u548c seatunnel \uff0cGoogle \u7814\u7a76\u4e86\u4e00\u4e0b\u90fd\u662f seatunnel \u5404\u79cd\u540a\u6253 datax \uff0c\u60f3\u7740 seatunnel \u6709 spark \u652f\u6301\u5c31\u91c7\u7528\u4e86\uff0c\u597d\uff0c\u4e07\u6076\u7684\u6e90\u5934\u5f00\u59cb\uff0cseatunnel \u8868\u9762\u770b\u8d77\u6765\u5404\u79cd\u9ad8\u5927\u4e0a\uff0c\u540e\u9762\u8fd8\u641e\u4e86\u81ea\u5df1\u7684\u5f15\u64ce\uff0c\u5b9e\u9645\u7528\u8d77\u6765\u554a\uff0c\u771f\u7684\u4e00\u5768\u72d7\u5c4e\uff0c\u57fa\u672c\u529f\u80fd\u4e0d\u597d\u597d\u641e\uff0c\u8f6f\u6587\u5230\u5904\u5439\uff0c\u5c31\u6700\u8fd1 v2.3.1 \u91cc\u7684 transform \u63d2\u4ef6\u7adf\u7136\u548c v2.3.0 \u4e0d\u517c\u5bb9\uff0cv2.1 \u5230 v2.3 \u7684\u4e0d\u517c\u5bb9\u6211\u5fcd\u4e86\uff0c0.01 \u7684\u5347\u7ea7\u80fd\u641e\u6210\u8fd9\u6837\u662f\u771f\u725b\u903c\u3002</p>\n<p>\u6587\u6863\u4f8b\u5b50\u4e0d\u597d\u597d\u5199\uff0c\u6bcf\u6b21\u51fa\u4e2a\u95ee\u9898\u8fd8\u5f97\u7ffb\u6e90\u7801\u770b\u600e\u4e48\u5904\u7406\uff0c\u6c38\u8fdc\u53ea\u4f1a\u5199\u4e2a hello world \u7ea7\u522b\u4f8b\u5b50\uff0c\u5c31\u8fd9\u7ea7\u522b\u4f8b\u5b50\u81ea\u5df1\u5199\u4e2a scala \u5c31\u641e\u5b9a\u4e86\u597d\u4f10\u3002</p>\n<p>\u5173\u952e\u662f\u8fd9\u7cdf\u7cd5\u7684\u9879\u76ee\u7adf\u7136\u8fd8\u80fd\u6df7\u8fdb Apache \uff0c\u674e\u5b89\u9707\u60ca\u8868\u793a\u4e0d\u7406\u89e3\u3002</p>\n<p>\u4e0a\u4e00\u6b21\u5403\u5c4e\u8fd8\u662f\u817e\u8baf\u5f00\u6e90\u7684\u6570\u636e\u5168\u5bb6\u6876\uff0c\u90a3\u662f\u771f\u6076\u5fc3\u3002</p>\n"
    }, 
    {
      "author": {
        "url": "https://www.v2ex.com/member/tonyjia87", 
        "name": "tonyjia87", 
        "avatar": "https://cdn.v2ex.com/gravatar/9eca2cae3995c6bab68bef70a015d404?s=73&d=retro"
      }, 
      "url": "https://www.v2ex.com/t/894551", 
      "title": "ETL \u65b9\u6848\u6709\u54ea\u4e9b", 
      "id": "https://www.v2ex.com/t/894551", 
      "date_published": "2022-11-11T10:07:13+00:00", 
      "content_html": "<ul>\n<li>\n<p>\u6d88\u8d39 kafka \u6570\u636e\u5199\u5230 CK \uff0cmysql</p>\n</li>\n<li>\n<p>\u4e0d\u60f3\u5199\u4ee3\u7801\u65b9\u5f0f\u5bf9\u63a5\u6570\u636e\u6e90</p>\n</li>\n<li>\n<p>\u770b\u770b\u793e\u533a\u6709\u4ec0\u4e48\u7ecf\u9a8c\uff0c\u907f\u514d\u8d70\u5f2f\u8def</p>\n</li>\n</ul>\n"
    }, 
    {
      "author": {
        "url": "https://www.v2ex.com/member/miscnote", 
        "name": "miscnote", 
        "avatar": "https://cdn.v2ex.com/gravatar/fa3a0431ee232cdeeccd0608eb5aed21?s=73&d=retro"
      }, 
      "url": "https://www.v2ex.com/t/838166", 
      "title": "\u4eca\u5929\u542c\u5230\u4e00\u4e2a\u7b11\u8bdd", 
      "id": "https://www.v2ex.com/t/838166", 
      "date_published": "2022-03-05T05:17:09+00:00", 
      "content_html": "<p>\u67d0\u516c\u53f8\uff08\u5e7f\u544a\u884c\u4e1a\uff09\u8fd0\u7ef4\u5c06 hadoop \u9ed8\u8ba4\u5b89\u88c5\u540e\u5c31\u6254\u7ed9\u6280\u672f\u7528\u3002\u9ed8\u8ba4\u5b89\u88c5\u7684 hdfs \u7684 data \u76ee\u5f55\u4f4d\u4e8e /tmp \u4e0b\u3002\u7136\u540e\u7cfb\u7edf\u4e00\u91cd\u542f\uff0c\u6240\u6709\u6570\u636e\u90fd\u6ca1\u4e86\u3002\u3002\u3002</p>\n"
    }, 
    {
      "author": {
        "url": "https://www.v2ex.com/member/tunzao", 
        "name": "tunzao", 
        "avatar": "https://cdn.v2ex.com/avatar/4ef4/2016/182412_large.png?m=1659852809"
      }, 
      "url": "https://www.v2ex.com/t/831157", 
      "title": "\u4efb\u52a1\u8c03\u5ea6\u7cfb\u7edf\u7cfb\u5217\u4e4b Airflow", 
      "id": "https://www.v2ex.com/t/831157", 
      "date_published": "2022-01-28T09:15:46+00:00", 
      "content_html": "<p><a href=\"http://tunzao.me/articles/airflow/\" rel=\"nofollow\">http://tunzao.me/articles/airflow/</a></p>\n"
    }, 
    {
      "author": {
        "url": "https://www.v2ex.com/member/NeezerGu", 
        "name": "NeezerGu", 
        "avatar": "https://cdn.v2ex.com/gravatar/fa5df1ba58dbb71a937ed15596cfc1b5?s=73&d=retro"
      }, 
      "url": "https://www.v2ex.com/t/807595", 
      "title": "100G \u5de6\u53f3\u6570\u636e\uff0c mysql \u7406\u8bba\u4e0a\u53ef\u80fd\u4f1a\u6bd4\u5927\u6570\u636e\u5f15\u64ce\u5feb\u5417\uff1f", 
      "id": "https://www.v2ex.com/t/807595", 
      "date_published": "2021-10-13T09:16:52+00:00", 
      "content_html": "<p>\u8ddf\u670b\u53cb\u95f2\u804a\u7a81\u7136\u60f3\u5230\uff0c\u56e0\u4e3a\u662f\u505a\u6570\u636e\u5206\u6790\u7684\uff0c\u5bf9\u5e95\u5c42\u5f15\u64ce\u5c31\u4e0d\u592a\u61c2\u5566\u3002</p>\n<p>\u6bd4\u5982\uff0c980pro 4 \u6761\u7ec4 raid 0 \uff08 YouTube \u4e0a\u6709\u89c6\u9891\uff0c<a href=\"https://www.youtube.com/watch?v=OCGguruZyrw&amp;t=413s\" rel=\"nofollow\">https://www.youtube.com/watch?v=OCGguruZyrw&amp;t=413s</a>\n\u751a\u81f3\u76f4\u63a5 256G \u5185\u5b58\uff0c\u62ff 120G \u5185\u5b58\u5f53\u786c\u76d8\uff08\u4e00\u65f6\u4e4b\u95f4\u5fd8\u4e86\u8fd9\u4e2a\u540d\u8bcd\uff09</p>\n<p>\u7528 mysql \u6216\u8005 clickhouse\uff0c\u5bf9\u6bd4 spark \u3001presto \u7b49\uff0c\u4f1a\u5b58\u5728\u524d\u8005\u66f4\u5feb\u7684\u53ef\u80fd\u5417\uff1f</p>\n"
    }, 
    {
      "author": {
        "url": "https://www.v2ex.com/member/zhangshaohan", 
        "name": "zhangshaohan", 
        "avatar": "https://cdn.v2ex.com/avatar/66d5/eebe/424937_large.png?m=1758694576"
      }, 
      "url": "https://www.v2ex.com/t/800736", 
      "title": "\u4e0d\u5f71\u54cd\u4e1a\u52a1\u7cfb\u7edf\u7684\u60c5\u51b5\u4e0b\u5982\u4f55\u505a\u5b9e\u65f6\u5927\u5c4f\u5c55\u793a\u5462\uff1f", 
      "id": "https://www.v2ex.com/t/800736", 
      "date_published": "2021-09-09T00:49:27+00:00", 
      "content_html": "<p>\u76ee\u524d\u9700\u8981\u505a\u4e2a\u5927\u5c4f\u5c55\u793a\uff0c\u4e0d\u80fd\u5bf9\u4e1a\u52a1\u7cfb\u7edf\u6709\u4fb5\u5165\u6027\uff0c\u9700\u8981\u5b9e\u73b0\u7c7b\u4f3c\u4e8e\u65b0\u589e\u4e00\u4ef6\u5546\u54c1\u5927\u5c4f\u4e0a\u6570\u5b57\u5c31\u5b9e\u65f6\u52a0 1\uff0c\u6570\u636e\u5e93 MySQL</p>\n<p>\u76ee\u524d\u8003\u8651\u662f\u901a\u8fc7 canal \u901a\u8fc7 binlog \u8ba2\u9605\u67d0\u51e0\u5f20\u8868\u6765\u5b9e\u73b0\u65b0\u589e\u901a\u77e5\u673a\u5236\uff0c\u6709\u6ca1\u6709\u4ec0\u4e48\u66f4\u597d\u7684\u529e\u6cd5\u5462\uff1f\u662f\u4e2a\u5c0f\u5de5\u7a0b\uff0c\u5468\u671f\u5f88\u77ed\u3002</p>\n<p>\u611f\u8c22\u5927\u5bb6\uff01\uff01\uff01</p>\n"
    }, 
    {
      "author": {
        "url": "https://www.v2ex.com/member/iriddle911", 
        "name": "iriddle911", 
        "avatar": "https://cdn.v2ex.com/avatar/14fc/6679/290993_large.png?m=1548033213"
      }, 
      "url": "https://www.v2ex.com/t/795438", 
      "title": "\u8feb\u4e8e\u538b\u529b\uff0c\u6c42\u4e2a\u597d\u7684\u5927\u6570\u636e\u8bfe\u7a0b", 
      "id": "https://www.v2ex.com/t/795438", 
      "date_published": "2021-08-12T13:21:09+00:00", 
      "content_html": "<p>\u672c\u4eba\u4ece\u4e8b to B \u552e\u524d\u5de5\u4f5c\uff0c\u6d89\u53ca\u5927\u6570\u636e\u9879\u76ee\u3002\u4f46\u5948\u4f55\u6ca1\u6709\u5927\u6570\u636e\u7684\u7814\u53d1\u3001\u9879\u76ee\u7ecf\u9a8c\uff0c\u81ea\u5b66\u5927\u6570\u636e\u603b\u4e0d\u5f97\u8981\u9886\u3002</p>\n<p>\u5728\u5404\u7f51\u7ad9\u627e\u5230\u7684\u8bfe\u7a0b\u57fa\u672c\u90fd\u662f\u9762\u5411\u7814\u53d1\u7684\uff0c\u4e0a\u6765\u5c31\u5f00\u59cb\u603c JAVA\uff0c\u5b66\u8d77\u6765\u592a\u96be\u4e86\u3002\u6545\u6c42\u5404\u4f4d\u5927\u4f6c\u63a8\u8350\uff0c\u80fd\u5feb\u901f\u4e86\u89e3\u5927\u6570\u636e\u6280\u672f\u751f\u6001\u7684\u4e66 /\u8bfe\u7a0b\uff0c\u5927\u81f4\u7c7b\u4f3c\u7ed9\u4ea7\u54c1\u7ecf\u7406\u8bb2\u6280\u672f\u8fd9\u6837\u7684\u7a0b\u5ea6\uff0c\u4e0d\u8981\u6c42\u638c\u63e1\u539f\u7406\uff0c\u77e5\u9053\u57fa\u672c\u5168\u666f\u8fc7\u7a0b\u5373\u53ef\u3002</p>\n<p>\u671f\u671b\uff1a\u80fd\u57fa\u672c\u4e86\u89e3\u5927\u6570\u636e /\u6570\u636e\u4e2d\u53f0\u7c7b\u4f3c\u9879\u76ee\u7684\u5168\u6d41\u7a0b\u6280\u672f\uff0c\u53ef\u4ee5\u548c RD \u8fdb\u884c\u5bf9\u8bdd\u3002</p>\n<p>\u672c\u4eba\u6280\u672f\u80cc\u666f\uff1a\u975e CS \u4e13\u4e1a\u51fa\u8eab\uff0c\u61c2\u4e00\u4e9b\u4e91\uff0c\u4f1a\u5199\u4e00\u4e9b\u524d\u7aef\uff0c\u77e5\u9053\u4f20\u7edf\u5de5\u7a0b\u5316\u7684\u7814\u53d1\u6d41\u7a0b\u3002\u6b63\u5728\u770b Hadoop \u7684\u5404\u79cd\u4e66\u3002</p>\n"
    }, 
    {
      "author": {
        "url": "https://www.v2ex.com/member/TalkWithKeyboard", 
        "name": "TalkWithKeyboard", 
        "avatar": "https://cdn.v2ex.com/avatar/28c2/7990/231971_large.png?m=1495468405"
      }, 
      "url": "https://www.v2ex.com/t/792769", 
      "title": "\u7845\u8c37\u6700\u706b\u7684\u6570\u636e\u5206\u6790\u5de5\u5177 dbt \u7a76\u7adf\u662f\u4ec0\u4e48\uff1f", 
      "id": "https://www.v2ex.com/t/792769", 
      "date_published": "2021-07-30T11:34:57+00:00", 
      "content_html": "<p>dbt \u63d0\u4f9b\u4e86\u5f88\u591a\u8f6f\u4ef6\u5f00\u53d1\u7684\u601d\u60f3\u548c\u5de5\u5177\u5230\u6570\u636e\u5206\u6790\u9886\u57df\uff0c\u6709\u4e86 dbt \u4ee5\u540e\u66f4\u591a\u7684\u5206\u6790\u5e08\u4f1a\u4e60\u60ef\u6210\u4e3a\u8fd9\u4e9b\u529f\u80fd\u7684\u4f7f\u7528\u8005\uff0c\u800c\u4e0d\u662f\u5f00\u53d1\u8005\u3002\u8f6f\u4ef6\u5de5\u7a0b\u5f00\u53d1\u8005\u5df2\u7ecf\u4e60\u60ef\u5728\u9700\u8981\u5199\u4e00\u4e2a\u65b0\u7684\u9700\u6c42\u7684\u65f6\u5019\uff0c\u76f4\u63a5\u4f7f\u7528\u4e00\u4e2a\u73b0\u6210\u7684\u5de5\u5177\uff0c\u4ed6\u4eec\u53ea\u9700\u8981\u7b80\u5355\u7684\u6784\u5efa\u5c31\u53ef\u4ee5\u4f7f\u7528\uff0c\u7136\u540e\u5c06\u4ed6\u4eec\u7684\u5de5\u4f5c\u518d\u8d21\u732e\u56de\u793e\u533a\u3002\u5728 dbt \u4e2d\u5206\u6790\u5e08\u4e5f\u53ef\u4ee5\u505a\u540c\u6837\u7684\u4e8b\u60c5\u3002</p>\n<p>\u8fd9\u4e24\u5e74\u56fd\u5916\u7684\u6570\u636e\u4ea7\u54c1 dbt \u975e\u5e38\u7684\u706b\uff0c\u56fd\u5185\u597d\u50cf\u8ba8\u8bba\u5f88\u5c11\uff0c\u6700\u8fd1\u7ffb\u8bd1\u4e86\u4e00\u7bc7 dbt \u521b\u59cb\u4eba\u4ecb\u7ecd dbt \u7684\u6587\u7ae0\u4e5f\u5199\u4e86\u4e00\u7bc7\u4f7f\u7528 dbt \u7684\u6559\u7a0b\u5e2e\u52a9\u5927\u5bb6\u4e86\u89e3 dbt\uff1a<a href=\"https://cndata.co/topic/7/%E7%A1%85%E8%B0%B7%E6%9C%80%E7%81%AB%E7%9A%84%E6%95%B0%E6%8D%AE%E5%88%86%E6%9E%90%E5%B7%A5%E5%85%B7dbt%E7%A9%B6%E7%AB%9F%E6%98%AF%E4%BB%80%E4%B9%88\" rel=\"nofollow\">\u6307\u8def</a></p>\n"
    }, 
    {
      "author": {
        "url": "https://www.v2ex.com/member/codeboy18", 
        "name": "codeboy18", 
        "avatar": "https://cdn.v2ex.com/avatar/3f57/60dc/280585_large.png?m=1614165693"
      }, 
      "url": "https://www.v2ex.com/t/757053", 
      "title": "\u4e00\u6587\u8bfb\u61c2 clickhouse \u96c6\u7fa4\u76d1\u63a7", 
      "id": "https://www.v2ex.com/t/757053", 
      "date_published": "2021-02-28T13:49:32+00:00", 
      "content_html": "<blockquote>\n<p>\u66f4\u591a\u7cbe\u5f69\u5185\u5bb9\uff0c\u8bf7\u5173\u6ce8\u5fae\u4fe1\u516c\u4f17\u53f7\uff1a<strong>\u540e\u7aef\u6280\u672f\u5c0f\u5c4b</strong> </p>\n</blockquote>\n<h1>\u4e00\u6587\u8bfb\u61c2 clickhouse \u96c6\u7fa4\u76d1\u63a7</h1>\n<p>\u5e38\u8a00\u9053\uff0c\u5175\u9a6c\u672a\u81f3\uff0c\u7cae\u8349\u5148\u884c\uff0c\u5728 clickhouse \u4e0a\u751f\u4ea7\u73af\u5883\u4e4b\u524d\uff0c\u6211\u4eec\u5c31\u5f97\u5236\u5b9a\u597d\u76f8\u5173\u7684\u76d1\u63a7\u65b9\u6848\uff0c\u5305\u62ec metric \u91c7\u96c6\u3001\u62a5\u8b66\u7b56\u7565\u3001\u56fe\u5f62\u5316\u62a5\u8868\u3002\u6709\u4e86\u5168\u9762\u6709\u6548\u7684\u76d1\u63a7\uff0c\u6211\u4eec\u5c31\u4eff\u4f5b\u62e5\u6709\u4e86\u5343\u91cc\u773c\u987a\u98ce\u8033\uff0c\u5bf9\u4e8e\u7ebf\u4e0a\u4efb\u4f55\u98ce\u5439\u8349\u52a8\u90fd\u80fd\u53ca\u65f6\u611f\u77e5\uff0c\u5728\u5fc5\u8981\u7684\u60c5\u51b5\u4e0b\u63d0\u524d\u4ecb\u5165\u4ee5\u907f\u514d\u7ebf\u4e0a\u6545\u969c\u3002  </p>\n<p>\u4e1a\u754c\u5e38\u7528\u7684\u76d1\u63a7\u65b9\u6848\u4e00\u822c\u662f\u57fa\u4e8e prometheus + grafana \u751f\u6001\u3002\u672c\u6587\u5c06\u4ecb\u7ecd\u7531 clickhouse-exporter(node-exporter) + prometheus + grafana \u7ec4\u6210\u7684\u76d1\u63a7\u65b9\u6848\u3002  </p>\n<p><img alt=\"clickhouse \u76d1\u63a7\u65b9\u6848\" class=\"embedded_image\" loading=\"lazy\" referrerpolicy=\"no-referrer\" rel=\"noreferrer\" src=\"https://backendhouse.github.io/images/clickhouse%E7%9B%91%E6%8E%A7%E6%96%B9%E6%A1%88.png\"/> </p>\n<p>\u4ee5\u4e0a\u4e3a\u76d1\u63a7\u65b9\u6848\u793a\u610f\u56fe  </p>\n<ul>\n<li>clickhouse-server \u4e2d\u6709 4 \u4e2a\u7cfb\u7edf\u8868\u4f1a\u8bb0\u5f55\u8fdb\u7a0b\u5185\u90e8\u7684\u6307\u6807\uff0c\u5206\u522b\u662f<code>system.metrics</code>\uff0c<code>system.asynchronous_metrics</code>, <code>system.events</code>\uff0c<code>system.parts</code> </li>\n<li>clickhuse-exporter \u662f\u4e00\u4e2a\u7528\u4e8e\u91c7\u96c6 clickhouse \u6307\u6807\u7684\u5f00\u6e90\u7ec4\u4ef6( <a href=\"https://github.com/ClickHouse/clickhouse_exporter)%EF%BC%8C%E5%AE%83%E4%BC%9A%E5%AE%9A%E6%97%B6%E6%9F%A5%E8%AF%A2\" rel=\"nofollow\">https://github.com/ClickHouse/clickhouse_exporter)\uff0c\u5b83\u4f1a\u5b9a\u65f6\u67e5\u8be2</a> clickhouse-server \u4e2d\u7684\u7cfb\u7edf\u8868\uff0c\u8f6c\u5316\u6210\u76d1\u63a7\u6307\u6807\uff0c\u5e76\u901a\u8fc7 HTTP \u63a5\u53e3\u66b4\u9732\u7ed9 prometheus.  </li>\n<li>node-exporter \u662f\u4e00\u4e2a\u7528\u4e8e\u91c7\u96c6\u786c\u4ef6\u548c\u64cd\u4f5c\u7cfb\u7edf\u76f8\u5173\u6307\u6807\u7684\u5f00\u6e90\u7ec4\u4ef6( <a href=\"https://github.com/prometheus/node_exporter)%E3%80%82\" rel=\"nofollow\">https://github.com/prometheus/node_exporter)\u3002</a> </li>\n<li>prometheus \u5b9a\u65f6\u6293\u53d6 clickhouse-exporter \u66b4\u9732\u7684\u6307\u6807\uff0c\u5e76\u5224\u65ad\u62a5\u8b66\u6761\u4ef6\u662f\u5426\u88ab\u89e6\u53d1\uff0c\u662f\u5219\u63a8\u9001\u5230 alert manager  </li>\n<li>DBA \u53ef\u901a\u8fc7 grafana \u770b\u677f\u5b9e\u65f6\u67e5\u770b\u5f53\u524d clickhouse \u96c6\u7fa4\u7684\u8fd0\u884c\u72b6\u6001  </li>\n<li>DBA \u53ef\u901a\u8fc7 alertmanager \u8bbe\u7f6e\u62a5\u8b66\u901a\u77e5\u65b9\u5f0f\uff0c\u5982\u90ae\u4ef6\u3001\u4f01\u4e1a\u5fae\u4fe1\u3001\u7535\u8bdd\u7b49\u3002  </li>\n</ul>\n<h2>1 \u90e8\u7f72\u4e0e\u914d\u7f6e</h2>\n<h3>1.1 clickhouse-server</h3>\n<p>\u6211\u4eec\u751f\u4ea7\u73af\u5883\u7248\u672c\u4e3a<code>20.3.8</code>\uff0c\u6309\u7167\u5b98\u65b9\u6587\u6863\u90e8\u7f72\u5373\u53ef\u3002  </p>\n<h3>1.2 clickhouse-exporter</h3>\n<p>clickhouse-exporter \u4e00\u822c\u4e0e clickhouse-server \u540c\u673a\u90e8\u7f72\u3002  </p>\n<p>\u9996\u5148\u4e0b\u8f7d\u6700\u65b0\u4ee3\u7801\u5e76\u7f16\u8bd1(\u9700\u9884\u5148\u5b89\u88c5 Go)  </p>\n<pre><code class=\"language-bash\">git clone https://github.com/ClickHouse/clickhouse_exporter  \ncd clickhouse_exporter  \ngo mod init  \ngo mod vendor  \ngo build   \nls ./clickhouse_exporter  \n</code></pre>\n<p>\u7136\u540e\u542f\u52a8  </p>\n<pre><code class=\"language-bash\">export CLICKHOUSE_USER=\"user\"  \nexport CLICKHOUSE_PASSWORD=\"password\"  \nnohup ./-scrape_uri=http://localhost:port/ &gt;nohup.log 2&gt;&amp;1 &amp;  \n</code></pre>\n<p>\u6700\u540e\u68c0\u67e5\u6307\u6807\u662f\u5426\u88ab\u6b63\u5e38\u91c7\u96c6\uff1a  </p>\n<pre><code>&gt; curl localhost:9116/metrics | head  \n# TYPE clickhouse_arena_alloc_bytes_total counter  \nclickhouse_arena_alloc_bytes_total 9.799096840192e+12  \n# HELP clickhouse_arena_alloc_chunks_total Number of ArenaAllocChunks total processed  \n# TYPE clickhouse_arena_alloc_chunks_total counter  \nclickhouse_arena_alloc_chunks_total 2.29782524e+08  \n# HELP clickhouse_background_move_pool_task Number of BackgroundMovePoolTask currently processed  \n# TYPE clickhouse_background_move_pool_task gauge  \nclickhouse_background_move_pool_task 0  \n# HELP clickhouse_background_pool_task Number of BackgroundPoolTask currently processed  \n</code></pre>\n<h3>1.3 node-exporter</h3>\n<p>node-exporter \u9700\u4e0e clickhouse-server \u540c\u673a\u90e8\u7f72  </p>\n<p>\u9996\u5148\u4e0b\u8f7d\u6700\u65b0\u4ee3\u7801\u5e76\u7f16\u8bd1  </p>\n<pre><code class=\"language-bash\">git clone https://github.com/prometheus/node_exporter  \nmake build  \nls ./node_exporter  \n</code></pre>\n<p>\u7136\u540e\u542f\u52a8  </p>\n<pre><code>nohup ./node_exporter &gt; nohup.log 2&gt;&amp;1 &amp;   \n</code></pre>\n<p>\u6700\u540e\u68c0\u67e5\u6307\u6807\u662f\u5426\u88ab\u6b63\u5e38\u91c7\u96c6  </p>\n<pre><code>&gt; curl localhost:9100/metrics  \n# HELP go_gc_duration_seconds A summary of the GC invocation durations.  \n# TYPE go_gc_duration_seconds summary  \ngo_gc_duration_seconds{quantile=\"0\"} 6.3563e-05  \ngo_gc_duration_seconds{quantile=\"0.25\"} 7.4746e-05  \ngo_gc_duration_seconds{quantile=\"0.5\"} 9.0556e-05  \ngo_gc_duration_seconds{quantile=\"0.75\"} 0.000110677  \ngo_gc_duration_seconds{quantile=\"1\"} 0.004362325  \ngo_gc_duration_seconds_sum 28.451282046  \ngo_gc_duration_seconds_count 223479  \n...  \n</code></pre>\n<h3>1.4 prometheus</h3>\n<p>\u4fee\u6539 prometheus \u914d\u7f6e\u6587\u4ef6\uff0c\u6dfb\u52a0 alertmanager \u5730\u5740\u3001clickhouse-exporter \u5730\u5740  </p>\n<p>prometheus.yml \u793a\u4f8b\u5982\u4e0b\uff1a  </p>\n<pre><code class=\"language-yaml\">global:  \n  scrape_interval:     15s # Set the scrape interval to every 15 seconds. Default is every 1 minute.  \n  evaluation_interval: 15s # Evaluate rules every 15 seconds. The default is every 1 minute.  \n  \n# Alertmanager configuration  \nalerting:  \n  alertmanagers:  \n  - static_configs:  \n    - targets:  \n      - alertmanager:9093  \n  \n# Load rules once and periodically evaluate them according to the global 'evaluation_interval'.  \nrule_files:  \n  - ./rules/*.rules  \n  \n# A scrape configuration containing exactly one endpoint to scrape:  \n# Here it's Prometheus itself.  \nscrape_configs:  \n  # The job name is added as a label `job=&lt;job_name&gt;` to any timeseries scraped from this config.  \n  - job_name: 'clickhouse'  \n  \n    # metrics_path defaults to '/metrics'  \n    # scheme defaults to 'http'.  \n    static_configs:  \n    - targets: ['clickhouseexporter1:9116', 'clickhouseexporter2:9116', ...]  \n</code></pre>\n<p>*.rules \u793a\u4f8b\u5982\u4e0b\uff1a  </p>\n<pre><code class=\"language-yaml\">groups:  \n - name: qps_too_high  \n   rules:  \n   - alert: clickhouse qps \u8d85\u51fa\u9608\u503c  \n     expr: rate(clickhouse_query_total[1m]) &gt; 100  \n     for: 2m  \n     labels:  \n      job: clickhouse-server  \n      severity: critical  \n      alertname: clickhouse qps \u8d85\u51fa\u9608\u503c  \n     annotations:  \n      summary: \"clickhouse qps \u8d85\u51fa\u9608\u503c\"  \n      description: \"clickhouse qps \u8d85\u8fc7\u9608\u503c(100), qps: {{ $value }}\"  \n</code></pre>\n<p>\u542f\u52a8 promethus  </p>\n<pre><code>nohup ./prometheus --config.file=/path/to/config --storage.tsdb.path=/path/to/storage --web.external-url=prometheus --web.enable-admin-api --web.enable-lifecycle --log.level=warn &gt;nohup.log 2&gt;&amp;1 &amp;   \n</code></pre>\n<p>\u6d4f\u89c8\u5668\u8f93\u5165<code>http://prometheus_ip:9090</code>\u68c0\u67e5 prometheus \u72b6\u6001  </p>\n<h3>1.5 alert manager</h3>\n<p>\u9996\u5148\u4fee\u6539\u914d\u7f6e\u6587\u4ef6  </p>\n<p>\u914d\u7f6e\u6587\u4ef6\u793a\u4f8b\u5982\u4e0b\uff1a  </p>\n<pre><code class=\"language-yaml\">route:  \n  receiver: 'default'  \n  group_by: ['service','project']  \n  \nreceivers:  \n- name: \"\u7535\u8bdd\"  \n  webhook_configs:  \n  - url: &lt;url&gt;  \n  \n- name: \"\u4f01\u4e1a\u5fae\u4fe1\"  \n  webhook_configs:  \n  - url: &lt;url&gt;  \n  \n- name: \"\u90ae\u7bb1\"  \n  webhook_configs:  \n  - url: &lt;url&gt;  \n</code></pre>\n<p>\u7136\u540e\u542f\u52a8  </p>\n<pre><code>nohup ./alertmanager --config.file=/path/to/config --log.level=warn &gt;nohup.log 2&gt;&amp;1 &amp;  \n</code></pre>\n<h3>1.6 grafana</h3>\n<p>\u5173\u4e8e clickhouse \u7684 dashboard \u6a21\u677f\u5df2\u7ecf\u6709\u5f88\u591a\uff0c\u5728\u8fd9\u91cc\u63a8\u8350\uff1a <a href=\"https://grafana.com/grafana/dashboards/882\" rel=\"nofollow\">https://grafana.com/grafana/dashboards/882</a> \u5c06\u5b83\u5bfc\u5165\u5230\u65b0\u5efa\u7684 grafana dashboard \u4e4b\u540e\uff0c\u5373\u53ef\u5f97\u5230\u6f02\u4eae\u7684 clickhouse \u96c6\u7fa4\u770b\u677f(\u53ef\u80fd\u9700\u8981\u5fae\u8c03)\u3002  </p>\n<p>\u53e6\u5916\u5efa\u8bae\u5b89\u88c5 clickhouse datasource \u63d2\u4ef6\u3002\u6709\u4e86\u8fd9\u4e2a\u63d2\u4ef6\u4fbf\u80fd\u5728 grafana \u4e2d\u914d\u7f6e clickhouse \u6570\u636e\u6e90\uff0c\u5e76\u901a\u8fc7 Clickhouse SQL \u914d\u7f6e\u56fe\u8868\uff0c\u8be6\u7ec6\u6587\u6863\u89c1\uff1a <a href=\"https://grafana.com/grafana/plugins/vertamedia-clickhouse-datasource\" rel=\"nofollow\">https://grafana.com/grafana/plugins/vertamedia-clickhouse-datasource</a> </p>\n<h2>2 \u91cd\u8981\u6307\u6807\u548c\u76d1\u63a7</h2>\n<p>\u6211\u4eec\u53ef\u4ee5\u770b\u5230\uff0c\u4e0d\u7ba1\u662f node-exporter \u8fd8\u662f clickhouse-exporter\uff0c\u5b83\u4eec\u7684\u6307\u6807\u79cd\u7c7b\u5f88\u591a\uff0c\u5927\u6982\u6709\u51e0\u767e\u4e2a\u3002\u6211\u4eec\u7684\u7b56\u7565\u662f\u6293\u5927\u653e\u5c0f\uff0c\u5bf9\u4e8e\u91cd\u8981\u7684\u6307\u6807\u624d\u8bbe\u7f6e\u62a5\u8b66\u7b56\u7565\u5e76\u521b\u5efa\u770b\u677f\u3002  </p>\n<p>\u4e0b\u9762\u5217\u4e3e\u4e00\u4e9b\u4e2a\u4eba\u89c9\u5f97\u6bd4\u8f83\u91cd\u8981\u7684\u6307\u6807   </p>\n<h3>2.1 \u7cfb\u7edf\u6307\u6807</h3>\n<p>\u7cfb\u7edf\u6307\u6807\u7531 node-exporter \u91c7\u96c6  </p>\n<table>\n<thead>\n<tr>\n<th>\u6307\u6807\u540d</th>\n<th>\u6307\u6807\u542b\u4e49</th>\n<th>\u62a5\u8b66\u7b56\u7565</th>\n<th>\u7b56\u7565\u542b\u4e49</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>node_cpu_seconds_total</td>\n<td>\u673a\u5668\u7d2f\u8ba1 cpu \u65f6\u95f4(\u5355\u4f4d s)</td>\n<td>100 * sum without (cpu) (rate(node_cpu_seconds_total{mode='user'}[5m])) / count without (cpu) (node_cpu_seconds_total{mode='user'}) &gt; 80</td>\n<td>\u7528\u6237\u6001 cpu \u5229\u7528\u7387\u5927\u4e8e 80%\u5219\u62a5\u8b66</td>\n</tr>\n<tr>\n<td>node_filesystem_size_bytes/node_filesystem_avail_bytes</td>\n<td>\u673a\u5668\u4e0a\u4e2a\u6587\u4ef6\u5206\u533a\u5bb9\u91cf /\u53ef\u7528\u5bb9\u91cf</td>\n<td>100 * (node_filesystem_size_bytes{mountpoint=\"/data\"} - node_filesystem_avail_bytes{mountpoint=\"/data\"}) / node_filesystem_size_bytes{mountpoint=\"/data\"} &gt; 80</td>\n<td>/data \u76d8\u5360\u7528\u8d85\u8fc7 80%\u5219\u62a5\u8b66</td>\n</tr>\n<tr>\n<td>node_load5</td>\n<td>5 \u5206\u949f load \u503c</td>\n<td>node_load5 &gt; 60</td>\n<td>5 \u5206\u949f load \u503c\u8d85\u8fc7 60 \u5219\u62a5\u8b66(\u53ef\u6839\u636e\u5177\u4f53\u60c5\u51b5\u8bbe\u7f6e\u9608\u503c)</td>\n</tr>\n<tr>\n<td>node_disk_reads_completed_total</td>\n<td>\u7d2f\u8ba1\u8bfb\u78c1\u76d8\u8bf7\u6c42\u6b21\u6570</td>\n<td>rate(node_disk_reads_completed_total[5m]) &gt; 200</td>\n<td>read iops \u8d85\u8fc7 200 \u5219\u62a5\u8b66</td>\n</tr>\n</tbody></table><h3>2.2 clickhouse \u6307\u6807</h3>\n<table>\n<thead>\n<tr>\n<th>\u6307\u6807\u540d</th>\n<th>\u6307\u6807\u542b\u4e49</th>\n<th>\u62a5\u8b66\u7b56\u7565</th>\n<th>\u7b56\u7565\u542b\u4e49</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>clickhouse_exporter_scrape_failures_total</td>\n<td>prometheus \u6293\u53d6 exporter \u5931\u8d25\u603b\u6b21\u6570</td>\n<td>increase(clickhouse_exporter_scrape_failures_total[5m]) &gt; 10</td>\n<td>prometheus \u6293\u53d6 export \u5931\u8d25\u6b21\u6570\u8d85\u8fc7\u9608\u503c\u5219\u62a5\u8b66\uff0c\u8bf4\u660e\u6b64\u65f6 ch \u670d\u52a1\u5668\u53ef\u80fd\u53d1\u751f\u5b95\u673a</td>\n</tr>\n<tr>\n<td>promhttp_metric_handler_requests_total</td>\n<td>exporter \u8bf7\u6c42 clickhouse \u5931\u8d25\u603b\u6b21\u6570</td>\n<td>increase(promhttp_metric_handler_requests_total{code=\"200\"}[2m]) == 0</td>\n<td>2 \u5206\u949f\u5185\u67e5\u8be2 clickhouse \u6210\u529f\u6b21\u6570\u4e3a\u96f6\u5219\u62a5\u8b66\uff0c\u8bf4\u660e\u6b64\u65f6\u67d0\u4e2a ch \u5b9e\u4f8b\u53ef\u80fd\u4e0d\u53ef\u7528</td>\n</tr>\n<tr>\n<td>clickhouse_readonly_replica</td>\n<td>ch \u5b9e\u4f8b\u4e2d\u5904\u4e8e\u53ea\u8bfb\u72b6\u6001\u7684\u8868\u4e2a\u6570</td>\n<td>clickhouse_readonly_replica &gt; 5</td>\n<td>ch \u4e2d\u53ea\u8bfb\u8868\u8d85\u8fc7 5 \u5219\u62a5\u8b66\uff0c\u8bf4\u660e\u6b64\u65f6 ch \u4e0e zk \u8fde\u63a5\u53ef\u80fd\u53d1\u751f\u5f02\u5e38</td>\n</tr>\n<tr>\n<td>clickhouse_query_total</td>\n<td>ch \u5df2\u5904\u7406\u7684 query \u603b\u6570</td>\n<td>rate(clickhouse_query_total[1m]) &gt; 30</td>\n<td>\u5355\u5b9e\u4f8b qps \u8d85\u8fc7 30 \u5219\u62a5\u8b66</td>\n</tr>\n<tr>\n<td>clickhouse_query</td>\n<td>ch \u4e2d\u6b63\u5728\u8fd0\u884c\u7684 query \u4e2a\u6570</td>\n<td>clickhouse_query &gt; 30</td>\n<td>\u5355\u5b9e\u4f8b\u5e76\u53d1 query \u6570\u8d85\u8fc7\u9608\u503c\u5219\u62a5\u8b66</td>\n</tr>\n<tr>\n<td>clickhouse_tcp_connection</td>\n<td>ch \u7684 TCP \u8fde\u63a5\u6570</td>\n<td>clickhouse_tcp_connection\t&gt; XXX</td>\n<td>\u7565</td>\n</tr>\n<tr>\n<td>clickhouse_http_connection</td>\n<td>ch \u7684 HTTP \u8fde\u63a5\u6570</td>\n<td>clickhouse_http_connection &gt; XXX</td>\n<td>\u7565</td>\n</tr>\n<tr>\n<td>clickhouse_zoo_keeper_request</td>\n<td>ch \u4e2d\u6b63\u5728\u8fd0\u884c\u7684 zk \u8bf7\u6c42\u6570</td>\n<td>clickhouse_zoo_keeper_request &gt; XXX</td>\n<td>\u7565</td>\n</tr>\n<tr>\n<td>clickhouse_replicas_max_queue_size</td>\n<td>ch \u4e2d zk \u526f\u672c\u540c\u6b65\u961f\u5217\u7684\u957f\u5ea6</td>\n<td>clickhouse_replicas_max_queue_size &gt; 100</td>\n<td>zk \u526f\u672c\u540c\u6b65\u961f\u5217\u957f\u5ea6\u8d85\u8fc7\u9608\u503c\u5219\u62a5\u8b66\uff0c\u8bf4\u660e\u6b64\u65f6\u526f\u672c\u540c\u6b65\u961f\u5217\u51fa\u73b0\u5806\u79ef</td>\n</tr>\n</tbody></table><h3>2.3 \u5176\u4ed6\u5e38\u7528 SQL</h3>\n<p>\u5728 clickhouse \u4e2d\uff0c\u6240\u6709\u88ab\u6267\u884c\u7684 Query \u90fd\u4f1a\u8bb0\u5f55\u5230<code>system.query_log</code>\u8868\u4e2d\u3002\u56e0\u6b64\u6211\u4eec\u53ef\u901a\u8fc7\u8be5\u8868\u76d1\u63a7\u96c6\u7fa4\u7684\u67e5\u8be2\u60c5\u51b5\u3002\u4ee5\u4e0b\u5217\u4e3e\u51e0\u79cd\u7528\u4e8e\u76d1\u63a7\u7684\u5e38\u7528 SQL \u3002\u4e3a\u4e86\u66f4\u65b9\u4fbf\u7684\u67e5\u770b\uff0c\u53ef\u6dfb\u52a0\u5230 grafana \u770b\u677f\u4e2d\u3002  </p>\n<p>\u6700\u8fd1\u67e5\u8be2  </p>\n<pre><code>SELECT   \n    event_time,   \n    user,   \n    query_id AS query,   \n    read_rows,   \n    read_bytes,   \n    result_rows,   \n    result_bytes,   \n    memory_usage,   \n    exception  \nFROM clusterAllReplicas('cluster_name', system, query_log)  \nWHERE (event_date = today()) AND (event_time &gt;= (now() - 60)) AND (is_initial_query = 1) AND (query NOT LIKE 'INSERT INTO%')  \nORDER BY event_time DESC  \nLIMIT 100  \n</code></pre>\n<p>\u6162\u67e5\u8be2  </p>\n<pre><code>SELECT   \n    event_time,   \n    user,   \n    query_id AS query,   \n    read_rows,   \n    read_bytes,   \n    result_rows,   \n    result_bytes,   \n    memory_usage,   \n    exception  \nFROM clusterAllReplicas('cluster_name', system, query_log)  \nWHERE (event_date = yesterday()) AND query_duration_ms &gt; 30000 AND (is_initial_query = 1) AND (query NOT LIKE 'INSERT INTO%')  \nORDER BY query_duration_ms desc  \nLIMIT 100  \n</code></pre>\n<p>Top10 \u5927\u8868  </p>\n<pre><code>SELECT   \n    database,   \n    table,   \n    sum(bytes_on_disk) AS bytes_on_disk  \nFROM clusterAllReplicas('cluster_name', system, parts)  \nWHERE active AND (database != 'system')  \nGROUP BY   \n    database,   \n    table  \nORDER BY bytes_on_disk DESC  \nLIMIT 10  \n</code></pre>\n<p>Top10 \u67e5\u8be2\u7528\u6237  </p>\n<pre><code>SELECT   \n    user,   \n    count(1) AS query_times,   \n    sum(read_bytes) AS query_bytes,   \n    sum(read_rows) AS query_rows  \nFROM clusterAllReplicas('cluster_name', system, query_log)  \nWHERE (event_date = yesterday()) AND (is_initial_query = 1) AND (query NOT LIKE 'INSERT INTO%')  \nGROUP BY user  \nORDER BY query_times DESC  \nLIMIT 10  \n</code></pre>\n<blockquote>\n<p>\u66f4\u591a\u7cbe\u5f69\u5185\u5bb9\uff0c\u8bf7\u626b\u7801\u5173\u6ce8\u5fae\u4fe1\u516c\u4f17\u53f7\uff1a<strong>\u540e\u7aef\u6280\u672f\u5c0f\u5c4b</strong>\u3002\u5982\u679c\u89c9\u5f97\u6587\u7ae0\u5bf9\u4f60\u6709\u5e2e\u52a9\u7684\u8bdd\uff0c\u8bf7\u591a\u591a\u5206\u4eab\u3001\u8f6c\u53d1\u3001\u5728\u770b\u3002          </p>\n</blockquote>\n<ul>\n<li><a href=\"https://mp.weixin.qq.com/s/dT3lCxkLliGgSRzSvXYxag\" rel=\"nofollow\">STL \u6e90\u7801\u5206\u6790--vector</a> </li>\n<li><a href=\"https://mp.weixin.qq.com/s/T7i0V2hcVU3HDO7DanPuGg\" rel=\"nofollow\">zookeeper client \u539f\u7406\u603b\u7ed3</a> </li>\n<li><a href=\"https://mp.weixin.qq.com/s/DQyUboTacgB3c_9iFi-aUw\" rel=\"nofollow\">redis \u5b9e\u73b0\u5206\u5e03\u5f0f\u9501</a> </li>\n<li><a href=\"https://mp.weixin.qq.com/s/S6cSgzHTjUOR9AhixPjnWA\" rel=\"nofollow\">\u63a8\u8350\u51e0\u4e2a\u597d\u7528\u7684\u6548\u7387\u795e\u5668</a> </li>\n<li><a href=\"https://mp.weixin.qq.com/s/sJoQAx6xk4lOlZMouayT-Q\" rel=\"nofollow\">C/C++\u5173\u952e\u5b57\u4e4b restrict</a> </li>\n<li><a href=\"https://mp.weixin.qq.com/s/qM_gf_aW_aKFhIK6PZTYEA\" rel=\"nofollow\">\u73b0\u4ee3 C++\u4e4b\u53f3\u503c\u8bed\u4e49</a> </li>\n<li><a href=\"https://mp.weixin.qq.com/s/LBnsNeUBf34rZrewlblzcA\" rel=\"nofollow\">Python \u4e71\u7801\u4e5d\u95ee</a> </li>\n<li><a href=\"https://mp.weixin.qq.com/s/Fux7QEtVeu-Zlde0aMnTNQ\" rel=\"nofollow\">Linux Shell \u811a\u672c\u653b\u7565\u8bfb\u4e66\u7b14\u8bb0</a></li>\n</ul>\n"
    }, 
    {
      "author": {
        "url": "https://www.v2ex.com/member/xiaomaisui", 
        "name": "xiaomaisui", 
        "avatar": "https://cdn.v2ex.com/gravatar/711df1b43b91206122fde9ed100dc0d0?s=73&d=retro"
      }, 
      "url": "https://www.v2ex.com/t/725158", 
      "title": "intellij \u8c03\u8bd5 spark \u7a0b\u5e8f\u53ef\u4ee5\u67e5\u770b\u5230 rdd \u6570\u636e\u5417", 
      "id": "https://www.v2ex.com/t/725158", 
      "date_published": "2020-11-14T04:45:19+00:00", 
      "content_html": "<p>\u5728\u7f16\u5199 spark \u7a0b\u5e8f\u7684\u65f6\u5019\u60f3\u901a\u8fc7 debug \u770b\u770b rdd \u7684\u8f6c\u6362\u64cd\u4f5c\u6b63\u786e\u4e0d\u6b63\u786e\uff0c\u4f46\u662f debug \u4ee5\u540e\u8c8c\u4f3c\u6ca1\u6709\u627e\u5230\uff0c\u6709\u6ca1\u6709\u5927\u4f6c\u77e5\u9053\u8fd9\u4e2a\u745e\u548c debug \u51fa\u6765 ?</p>\n"
    }, 
    {
      "author": {
        "url": "https://www.v2ex.com/member/oahebky", 
        "name": "oahebky", 
        "avatar": "https://cdn.v2ex.com/avatar/2e9e/c57d/429363_large.png?m=1587021641"
      }, 
      "url": "https://www.v2ex.com/t/715520", 
      "title": "talend \u5728\u4e1a\u5185\uff08\u6570\u636e\u5206\u6790\u3001ETL\u3001BI\uff09\u662f\u4ec0\u4e48\u60c5\u51b5\uff1f\u597d\u7528\uff1f Java \u7684\u5b9e\u8d28\u6027\u6807\u51c6\uff1f\u6210\u4e3a\u5751\u4e86\uff0c\u88ab\u55e4\u4e4b\u4ee5\u9f3b\uff1f etc...???", 
      "id": "https://www.v2ex.com/t/715520", 
      "date_published": "2020-10-16T02:49:54+00:00", 
      "content_html": "<h2>[ talend \u7b80\u4ecb]</h2>\n<h3>1. \u4ec0\u4e48\u662f Talend</h3>\n<p>Talend Open Studio \u662f Talend \uff08\u62d3\u84dd\uff09\u516c\u53f8\u5f00\u53d1\u7684\u4e00\u4e2a\u6570\u636e\u96c6\u6210\u7684\u6570\u636e ETL \u8f6f\u4ef6\uff0c\u53ef\u4ee5\u7b80\u5316\u6570\u636e\u5904\u7406\u6d41\u7a0b\uff0c\u964d\u4f4e\u5165\u95e8\u95e8\u69db\uff0c\u4e0d\u9700\u8981\u638c\u63e1\u4e13\u4e1a\u7684 ETL \u77e5\u8bc6\uff0c\u4ec5\u4ec5\u901a\u8fc7 web \u754c\u9762\u548c\u7b80\u5355\u7684\u7ec4\u4ef6\u62d6\u62fd\u5c31\u53ef\u5b9e\u73b0\u6570\u636e\u5904\u7406\u3002\u53ef\u4ee5\u534f\u52a9\u4f01\u4e1a\u5229\u7528\u66f4\u591a\u6570\u636e\uff0c\u4e0d\u65ad\u63d0\u9ad8\u5176\u6570\u636e\u7684\u53ef\u7528\u6027\u3001\u53ef\u9760\u6027\u4ee5\u53ca\u6709\u7528\u6027\u3002BI \u5de5\u5177 Talend Open Studio \u529f\u80fd\u5f3a\u5927\uff0c\u53ef\u4ee5\u540c\u6b65\u591a\u79cd\u6570\u636e\u5e93\uff0c\u53ef\u4ee5\u6e05\u6d17\u3001\u7b5b\u9009\u3001java \u4ee3\u7801\u5904\u7406\u6570\u636e\u3001\u6570\u636e\u5bfc\u5165\u5bfc\u51fa\uff0c\u5185\u8054\u67e5\u8be2\u591a\u79cd\u6570\u636e\u5e93\uff0c\u4ee5\u4e0b\u7b80\u79f0 talend \u3002</p>\n<p>\u6982\u51b5\u6765\u8bf4\uff0ctalend \u7279\u70b9\u4e3b\u8981\u6709\u4ee5\u4e0b\u51e0\u70b9\uff1a</p>\n<p>\u6570\u636e\u6e90\uff1a\u5404\u79cd\u5e38\u7528\u6570\u636e\u5e93\uff08 mysql oracle hive \uff09\uff0c\u6587\u4ef6\u7b49\u3002</p>\n<ul>\n<li>\n<p>\u901f\u5ea6\uff1a\u9700\u8981\u624b\u5de5\u8c03\u6574\uff0c\u5bf9\u7279\u5b9a\u6570\u636e\u6e90\u6709\u4f18\u5316\u3002</p>\n</li>\n<li>\n<p>\u90e8\u7f72\uff1a\u521b\u5efa java \u6216 perl \u6587\u4ef6\uff0c\u5e76\u901a\u8fc7\u64cd\u4f5c\u7cfb\u7edf\u8c03\u5ea6\u5de5\u5177\u6765\u8fd0\u884c\u3002</p>\n</li>\n<li>\n<p>\u6613\u7528\u6027\uff1a\u6709 GUI \u56fe\u5f62\u754c\u9762\u4f46\u662f\u4ee5 Eclipse \u7684\u63d2\u4ef6\u65b9\u5f0f\u63d0\u4f9b\u3002</p>\n</li>\n</ul>\n<hr/>\n<p>\u7b49\u5728\u4f7f\u7528 talend \u7684\u670b\u53cb\u8bf4\u8bf4\u770b\u3002</p>\n"
    }, 
    {
      "author": {
        "url": "https://www.v2ex.com/member/zealinux", 
        "name": "zealinux", 
        "avatar": "https://cdn.v2ex.com/gravatar/97cec5ffbda3f47b2cd453cf10b78244?s=73&d=retro"
      }, 
      "url": "https://www.v2ex.com/t/657850", 
      "date_modified": "2020-03-31T02:51:59+00:00", 
      "content_html": "<p>\u4e0d\u662f\u81ea\u5df1\u7528\uff0c\uff08\u4e0d\u8981\u7f51\u76d8\uff09\n\u60f3\u79c1\u6709\u5316\u90e8\u7f72\u516c\u53f8\u7684\u670d\u52a1\u3002\n\u5176\u4e2d\u6587\u4ef6\u5b58\u50a8\u662f\u7528\u7684 s3\uff0c\n\u73b0\u5728\u60f3\u627e\u4e2a\u5f00\u6e90\u7684\u7c7b\u4f3c\u7684\u670d\u52a1\uff0c\u81ea\u5df1\u642d\u5efa</p>\n<p>\u4e3b\u8981\u662f\u4e0a\u4f20 API \u63a5\u53e3\uff0c\u6709 bucket\uff0c\n\u4e0a\u4f20\u4e2a\u56fe\u7247\uff0c\u7ed9\u51fa\u4e2a\u516c\u5f00 http \u5730\u5740\uff0c\n\u5982\u679c\u6709\u6743\u9650\u8ba4\u8bc1\u5c31\u66f4\u597d\u4e86\u3002</p>\n<hr/>\n<p>\u611f\u89c9 nginx \u5e94\u8be5\u4e5f\u662f\u53ef\u4ee5\u505a\u8fd9\u6837\u7684\u4e8b\u60c5\uff0c\n\u4f46\u662f\u8981\u8003\u8651\u5230\u78c1\u76d8\u53ef\u80fd\u4f1a\u7206\u6389\uff0c\n\u4f46\u5982\u679c\u4e00\u5f00\u59cb\u5c31\u7533\u8bf7\u5f88\u5927\u7684\u78c1\u76d8\uff0c\u53c8\u5f88\u6d6a\u8d39\u3002\n\uff08\u542c\u522b\u4eba\u8bf4\uff0c\u8981\u7528 HDFS \u8fd9\u7c7b\u7684\u4e1c\u897f\u3002\uff09</p>\n<p>\u5927\u5bb6\u6709\u4ec0\u4e48\u53ef\u4ee5\u63a8\u8350\u7684\uff1f\n\uff08\u4e00\u5b9a\u8981\u652f\u6301 Docker \u90e8\u7f72\uff09</p>\n", 
      "date_published": "2020-03-31T02:51:39+00:00", 
      "title": "\u6709\u4ec0\u4e48\u81ea\u5efa [\u6587\u4ef6\u670d\u52a1] \u63a8\u8350\uff0c\u7528\u6765\u81ea\u5efa\u66ff\u4ee3 S3 \u670d\u52a1\uff1f", 
      "id": "https://www.v2ex.com/t/657850"
    }, 
    {
      "author": {
        "url": "https://www.v2ex.com/member/JohnXu20151211", 
        "name": "JohnXu20151211", 
        "avatar": "https://cdn.v2ex.com/gravatar/58b49211768add9b479f99100faba14f?s=73&d=retro"
      }, 
      "url": "https://www.v2ex.com/t/652310", 
      "title": "\u600e\u4e48\u52a0\u6df1\u5927\u6570\u636e\u6280\u80fd\uff0c\u8fdb\u5165\u5927\u5382\u3002\u597d\u8ff7\u832b,\u6c42\u6307\u70b9", 
      "id": "https://www.v2ex.com/t/652310", 
      "date_published": "2020-03-12T13:28:58+00:00", 
      "content_html": "<p>\u505a\u5f00\u53d1 4 \u5e74\u4e86\u5427\uff0c\u73b0\u5728\u662f\u505a\u5927\u6570\u636e\u76f8\u5173\u7684\uff0c\u516c\u53f8\u662f\u505a\u4e09\u65b9\u652f\u4ed8\u7684\u3002\u5728\u8fd9\u5bb6\u516c\u53f8\u5df2\u7ecf\u5446\u4e86\u6709 1 \u5e74\u534a\u4e86\uff0c\u4ece\u4e00\u5f00\u59cb\u6765\u7684\u65f6\u5019\u5e76\u4e0d\u662f\u505a\u5927\u6570\u636e\u505a\u7684 java\uff0c\u540e\u6765\u5f00\u59cb\u505a\u4e86\u5927\u6570\u636e\u628a\u516c\u53f8\u4e4b\u524d\u7684\u5927\u6570\u636e\u5e73\u53f0\u91cd\u6784\u505a\u4e86\u4e00\u7248\uff0c\u4f46\u662f\u73b0\u5728\u8d8a\u5446\u7740\u8d8a\u53d1\u73b0\u5b66\u4e0d\u5230\u4ec0\u4e48\u4e1c\u897f\uff0c\u56e0\u4e3a\u5176\u5b9e\u516c\u53f8\u6ca1\u6709\u51e0\u4eba\u771f\u6b63\u61c2\u5927\u6570\u636e\u7684\uff0c\u5305\u62ec\u6211\u4eec\u5c0f\u7ec4\u4e5f\u4e00\u6837\u3002\u73b0\u5728\u516c\u53f8\u662f\u60f3\u505a\u5173\u4e8e\u5b9e\u65f6\u7684\u4e1c\u897f\uff0c\u4f46\u5176\u5b9e\u6240\u6709\u4eba\u90fd\u6ca1\u6709\u4e00\u4e2a\u601d\u8def\u8be5\u600e\u4e48\u505a\uff0c\u505a\u4ec0\u4e48\uff0c\u6bd5\u7adf\u6211\u4eec\u6570\u636e\u6bd4\u8f83\u5c40\u9650\u3002\u6211\u5176\u5b9e\u7279\u522b\u60f3\u80fd\u8d81\u5e74\u8f7b\u8fdb\u5927\u5382\u5386\u7ec3\u5386\u7ec3\uff0c\u5b66\u4e9b\u6280\u672f\u3002\u4f46\u662f\u672c\u8eab\u5b66\u5386\u4e0d\u9ad8\uff0c\u518d\u52a0\u4e0a\u73b0\u5728\u516c\u53f8\u7684\u5927\u6570\u636e\u6ca1\u6709\u4ec0\u4e48\u6280\u672f\u542b\u91cf\uff0c\u6211\u8ba4\u4e3a\u5927\u6570\u636e\u7684\u771f\u6b63\u7684\u6280\u672f\u5e94\u8be5\u5c31\u662f\u6570\u636e\u4e0b\u94bb\u3001\u5b9e\u65f6\u3001\u6316\u6398\u3001\u5206\u6790\u3001\u673a\u5668\u5b66\u4e60\u7b49\u7b49\u8fd9\u4e9b\u3002\u522b\u4eba\u4e00\u95ee\u4ec0\u4e48\u5b9e\u65f6\u3001\u6570\u636e\u4e0b\u94bb\u3001\u6570\u636e\u6316\u6398\u4ec0\u4e48\u7684\u6211\u4eec\u90fd\u6ca1\u6709\u505a\uff0c\u5176\u5b9e\u5c31\u76f8\u5f53\u4e8e\u4e2a\u67e5\u8be2\u3002\u6240\u4ee5\u60f3\u53bb\u5927\u5382\u53c8\u6ca1\u6709\u62ff\u5f97\u51fa\u624b\u7684\u9879\u76ee\uff0c\u73b0\u5728\u53c8\u5b66\u4e0d\u5230\u4ec0\u4e48\uff0c\u56e0\u4e3a\u6211\u611f\u89c9\u5c31\u6211\u4eec\u73b0\u5728\u80fd\u529b\u4e0d\u592a\u53ef\u80fd\u505a\u51fa\u76f8\u5bf9\u6709\u4e9b\u6280\u672f\u542b\u91cf\u7684\u4e1c\u897f\u3002\u81ea\u5df1\u6709\u65f6\u95f4\u5c31\u5404\u79cd\u5237\u7b97\u6cd5\u9898\uff0c\u770b\u5927\u6570\u636e\u7684\u89c6\u9891\u3002\u4f46\u662f\u50cf spark\u3001flink \u7b49\u7b49\u8fd9\u4e9b\u8fd8\u662f\u8981\u5b9e\u8df5\uff0c\u6211\u81ea\u5df1\u4e5f\u642d\u4e86\u4e2a\u5b66\u4e60\u7684\u96c6\u7fa4\uff0c\u4f46\u662f\u4e5f\u4e0d\u77e5\u9053\u81ea\u5df1\u80fd\u7528\u8fd9\u4e9b\u505a\u4e9b\u4ec0\u4e48\u80fd\u9762\u8bd5\u7684\u4e1c\u897f\u3002 \u6073\u8bf7\u5404\u4f4d\u5927\u4f6c\u8bf4\u51fa\u4f60\u4eec\u7684\u770b\u6cd5\uff0c\u63d0\u51fa\u4f60\u4eec\u7684\u5efa\u8bae\u3002</p>\n"
    }, 
    {
      "author": {
        "url": "https://www.v2ex.com/member/zealinux", 
        "name": "zealinux", 
        "avatar": "https://cdn.v2ex.com/gravatar/97cec5ffbda3f47b2cd453cf10b78244?s=73&d=retro"
      }, 
      "url": "https://www.v2ex.com/t/646403", 
      "date_modified": "2020-02-21T08:39:58+00:00", 
      "content_html": "<p>\u5e7f\u544a\u7cfb\u7edf\u4f1a\u8ddf\u8e2a\u4e00\u4e9b\u6570\u636e\uff08\u7c7b\u4f3c log \uff09\uff0c\u8fd9\u4e9b\u6570\u636e\u4e13\u95e8\u7528\u6765\u505a\u7edf\u8ba1\u7684\u3002</p>\n<p>\u4f46\u662f\u7531\u4e8e\u4ee5\u540e\u53ef\u80fd\u91cf\u6bd4\u8f83\u5927\uff0c\u4e0d\u60f3\u7740\u5b58\u6570\u636e\u5e93\uff08 MySQL \uff09\u3002</p>\n<p>\u73b0\u5728\u4e00\u822c\u662f\u600e\u4e48\u597d\u505a\u6cd5\uff1f</p>\n<hr/>\n<p>\u6bd4\u5982\uff0c\u73b0\u5728\u60f3\u5230\u7684\u6bd4\u8f83\u7b28\u7684\u65b9\u6cd5\uff1a</p>\n<p>\u7cfb\u7edf\u4f1a\u6709\u9891\u7e41\u8ddf\u8e2a\u6570\u636e\u4f20\u8fc7\u6765\uff0c\n\u90a3\u4e48\u6211\u5c31\u5728 DB \u91cc [\u8ddf\u8e2a\u65e5\u5fd7\u8868] \u91cc\u65b0\u5efa\u4e00\u6761\u8bb0\u5f55\u3002</p>\n<p>\u7136\u540e\u5b9a\u65f6\u6bcf 10 \u5206\u949f\u6c47\u603b\u4e00\u4e9b\u3002\u8bb0\u5f55\u5728 [\u5206\u5c0f\u65f6\u7edf\u8ba1\u8868] \u91cc\u3002\n\u7136\u540e\u518d\u5b9a\u65f6\u4ece [\u5206\u5c0f\u65f6\u7edf\u8ba1\u8868] \u7b97\u51fa\u603b\u91cf\uff0c\u5b58\u653e [\u603b\u91cf\u8868] \u7684\u3002</p>\n<p>\u73b0\u5728\u662f 3 \u5f20\u8868\u3002</p>\n<pre><code>A - \u8ddf\u8e2a\u8bb0\u5f55\u8868\nB - \u5206\u5c0f\u65f6\u7edf\u8ba1\u8868\nC - \u6bcf\u4e2a\u5e7f\u544a\u7684\u7edf\u8ba1\u603b\u91cf\u8868\n</code></pre>\n<hr/>\n<p>\u611f\u89c9\u8fd9\u79cd\u65b9\u6cd5\u4e0d\u597d\uff0c\nA \u8868\u5e94\u8be5\u7b97\u662f\u975e\u4e3b\u4e1a\u52a1\u7684\uff0c\u4e0d\u5e94\u8be5\u4e0e\u4e1a\u52a1\u8868\u4eec\u653e\u5728\u4e00\u8d77\u3002\nA \u8868\u4e5f\u4f1a\u8d8a\u6765\u8d8a\u5e9e\u5927\u3002\uff08\u5927\u8868\u505a\u7edf\u8ba1\u7684\u8bdd\uff0c\u53ef\u80fd\u4f1a\u628a MySQL \u641e\u5d29\u6e83\u6389\uff09\n\u5176\u4f59\u4e24\u4e2a\u8868\u662f\u7531\u4e8e A \u6765\u7b97\u51fa\u6765\u7684\u3002</p>\n<p>\u5e94\u8be5\u4f1a\u6709\u66f4\u597d\u7684\u89e3\u51b3\u65b9\u6cd5\uff0c\u6c42\u6559</p>\n", 
      "date_published": "2020-02-21T08:39:46+00:00", 
      "title": "[\u7cfb\u7edf\u8bbe\u8ba1] \u5e7f\u544a\u7cfb\u7edf\u7684\u7edf\u8ba1\u6570\u636e\u8be5\u600e\u4e48\u8bbe\u8ba1\uff1f", 
      "id": "https://www.v2ex.com/t/646403"
    }, 
    {
      "author": {
        "url": "https://www.v2ex.com/member/PingCAP", 
        "name": "PingCAP", 
        "avatar": "https://cdn.v2ex.com/avatar/daac/3644/162717_large.png?m=1536739465"
      }, 
      "url": "https://www.v2ex.com/t/634414", 
      "title": "\u590d\u6742\u5206\u5e03\u5f0f\u67b6\u6784\u4e0b\u7684\u8ba1\u7b97\u6cbb\u7406\u4e4b\u8def", 
      "id": "https://www.v2ex.com/t/634414", 
      "date_published": "2020-01-02T05:27:09+00:00", 
      "content_html": "<blockquote>\n<p>\u5206\u5e03\u5f0f\u6280\u672f\u7684\u53d1\u5c55\uff0c\u6df1\u523b\u5730\u6539\u53d8\u4e86\u6211\u4eec\u7f16\u7a0b\u7684\u6a21\u5f0f\u548c\u601d\u8003\u8f6f\u4ef6\u7684\u6a21\u5f0f\u3002\u503c 2019 \u5c81\u672b\uff0cPingCAP \u8054\u5408 InfoQ \u5171\u540c\u7b56\u5212\u51fa\u54c1\u201c\u5206\u5e03\u5f0f\u7cfb\u7edf\u524d\u6cbf\u6280\u672f \u201d\u4e13\u9898\uff0c \u9080\u8bf7\u4f17\u591a\u6280\u672f\u56e2\u961f\u5171\u540c\u53c2\u4e0e\uff0c\u4e00\u8d77\u63a2\u7d22\u8fd9\u4e2a\u53e4\u8001\u9886\u57df\u7684\u65b0\u751f\u673a\u3002\u672c\u6587\u51fa\u81ea\u5fae\u4f17\u94f6\u884c\u5927\u6570\u636e\u5e73\u53f0\u8d1f\u8d23\u4eba\u90b8\u5e05\u3002</p>\n</blockquote>\n<p>\u5728\u5f53\u524d\u7684\u590d\u6742\u5206\u5e03\u5f0f\u67b6\u6784\u73af\u5883\u4e0b\uff0c\u670d\u52a1\u6cbb\u7406\u5df2\u7ecf\u5927\u884c\u5176\u9053\u3002\u4f46\u76ee\u5149\u5f80\u4e0b\u4e00\u5c42\uff0c\u4ece\u4e0a\u5c42 APP\u3001Service\uff0c\u5230\u5e95\u5c42\u8ba1\u7b97\u5f15\u64ce\u8fd9\u4e00\u5c42\u9762\uff0c\u5374\u8fd8\u662f\u5404\u4e2a\u5f15\u64ce\u5404\u81ea\u4e3a\u653f\uff0cClient-Server \u6a21\u5f0f\u7d27\u8026\u5408\u6ee1\u5929\u98de\u7684\u60c5\u51b5\u3002\u5982\u4f55\u505a\u597d\u201c\u8ba1\u7b97\u6cbb\u7406\u201d\uff0c\u8ba9\u590d\u6742\u73af\u5883\u4e0b\u5404\u79cd\u7c7b\u578b\u7684\u5927\u91cf\u8ba1\u7b97\u4efb\u52a1\uff0c\u90fd\u80fd\u66f4\u7b80\u6d01\u3001\u7075\u6d3b\u3001\u6709\u5e8f\u3001\u53ef\u63a7\u7684\u63d0\u4ea4\u6267\u884c\uff0c\u548c\u4fdd\u969c\u6210\u529f\u8fd4\u56de\u7ed3\u679c\uff1f\u8ba1\u7b97\u4e2d\u95f4\u4ef6 Linkis \u5c31\u662f\u4e0a\u8ff0\u95ee\u9898\u7684\u6700\u4f73\u5b9e\u8df5\u3002</p>\n<h2>\u590d\u6742\u5206\u5e03\u5f0f\u67b6\u6784\u73af\u5883\u4e0b\u7684\u8ba1\u7b97\u6cbb\u7406\u6709\u4ec0\u4e48\u95ee\u9898\uff1f</h2>\n<h3>\u4ec0\u4e48\u662f\u590d\u6742\u5206\u5e03\u5f0f\u67b6\u6784\u73af\u5883\uff1f</h3>\n<p>\u5206\u5e03\u5f0f\u67b6\u6784\uff0c\u6307\u7684\u662f\u7cfb\u7edf\u7684\u7ec4\u4ef6\u5206\u5e03\u5728\u901a\u8fc7\u7f51\u7edc\u76f8\u8fde\u7684\u4e0d\u540c\u8ba1\u7b97\u673a\u4e0a\uff0c\u7ec4\u4ef6\u4e4b\u95f4\u901a\u8fc7\u7f51\u7edc\u4f20\u9012\u6d88\u606f\u8fdb\u884c\u901a\u4fe1\u548c\u534f\u8c03\uff0c\u534f\u540c\u5b8c\u6210\u67d0\u4e00\u76ee\u6807\u3002\u4e00\u822c\u6765\u8bf4\u6709\u6c34\u5e73\uff08\u96c6\u7fa4\u5316\uff09\u548c\u5782\u76f4\uff08\u529f\u80fd\u6a21\u5757\u5207\u5206\uff09\u4e24\u4e2a\u62c6\u5206\u65b9\u5411\uff0c\u4ee5\u89e3\u51b3\u9ad8\u5185\u805a\u4f4e\u8026\u5408\u3001\u9ad8\u5e76\u53d1\u3001\u9ad8\u53ef\u7528\u7b49\u65b9\u9762\u95ee\u9898\u3002</p>\n<p>\u591a\u4e2a\u5206\u5e03\u5f0f\u67b6\u6784\u7684\u7cfb\u7edf\uff0c\u7ec4\u6210\u5206\u5e03\u5f0f\u7cfb\u7edf\u7fa4\uff0c\u5c31\u5f62\u6210\u4e86\u4e00\u4e2a\u76f8\u5bf9\u590d\u6742\u7684\u5206\u5e03\u5f0f\u67b6\u6784\u73af\u5883\u3002\u901a\u5e38\u5305\u542b\u591a\u79cd\u4e0a\u5c42\u5e94\u7528\u670d\u52a1\uff0c\u591a\u79cd\u5e95\u5c42\u57fa\u7840\u8ba1\u7b97\u5b58\u50a8\u5f15\u64ce\u3002\u5982\u4e0b\u56fe 1 \u6240\u793a\uff1a</p>\n<p><img alt=\"\" src=\"https://imgconvert.csdnimg.cn/aHR0cHM6Ly91cGxvYWQtaW1hZ2VzLmppYW5zaHUuaW8vdXBsb2FkX2ltYWdlcy81NDI2NzctNmVmMjk0YTA2YTRkNWFiZA?x-oss-process=image/format,png\"/></p>\n&lt;center&gt;\u56fe 1&lt;/center&gt;\n<h3>\u4ec0\u4e48\u662f\u8ba1\u7b97\u6cbb\u7406\uff1f</h3>\n<p>\u5c31\u50cf\u300a\u5fae\u670d\u52a1\u8bbe\u8ba1\u300b\u4e00\u4e66\u4e2d\u63d0\u5230\u7684\uff0c\u5982\u540c\u57ce\u5e02\u89c4\u5212\u5e08\u5728\u9762\u5bf9\u4e00\u5ea7\u5e9e\u5927\u3001\u590d\u6742\u4e14\u4e0d\u65ad\u53d8\u5316\u7684\u57ce\u5e02\u65f6\uff0c\u6240\u9700\u8981\u505a\u7684\u89c4\u5212\u3001\u8bbe\u8ba1\u548c\u6cbb\u7406\u4e00\u6837\uff0c\u5e9e\u5927\u590d\u6742\u7684\u8f6f\u4ef6\u7cfb\u7edf\u73af\u5883\u4e2d\u7684\u5404\u79cd\u533a\u57df\u3001\u5143\u7d20\u3001\u89d2\u8272\u548c\u5173\u7cfb\uff0c\u4e5f\u9700\u8981\u6574\u6cbb\u548c\u7ba1\u7406\uff0c\u4ee5\u4f7f\u5176\u4ee5\u4e00\u79cd\u66f4\u7b80\u6d01\u3001\u4f18\u96c5\u3001\u6709\u5e8f\u3001\u53ef\u63a7\u7684\u65b9\u5f0f\u534f\u540c\u8fd0\u4f5c\uff0c\u800c\u4e0d\u662f\u53d8\u6210\u4e00\u56e2\u4e71\u9ebb\u3002</p>\n<p>\u5728\u5f53\u524d\u7684\u590d\u6742\u5206\u5e03\u5f0f\u67b6\u6784\u73af\u5883\u4e0b\uff0c\u5927\u91cf APP\u3001Service \u95f4\u7684\u901a\u4fe1\u3001\u534f\u8c03\u548c\u7ba1\u7406\uff0c\u5df2\u7ecf\u6709\u4e86\u4ece SOA \uff08 Service-Oriented Architecture \uff09\u5230\u5fae\u670d\u52a1\u7684\u6210\u719f\u7406\u5ff5\uff0c\u53ca\u4ece ESB \u5230 Service Mesh \u7684\u4f17\u591a\u5b9e\u8df5\uff0c\u6765\u5b9e\u73b0\u5176\u4ece\u670d\u52a1\u6ce8\u518c\u53d1\u73b0\u3001\u914d\u7f6e\u7ba1\u7406\u3001\u7f51\u5173\u8def\u7531\uff0c\u5230\u6d41\u63a7\u7194\u65ad\u3001\u65e5\u5fd7\u76d1\u63a7\u7b49\u4e00\u7cfb\u5217\u5b8c\u6574\u7684\u670d\u52a1\u6cbb\u7406\u529f\u80fd\u3002\u670d\u52a1\u6cbb\u7406\u6846\u67b6\u7684\u201c\u4e2d\u95f4\u4ef6\u201d\u5c42\u8bbe\u8ba1\uff0c\u53ef\u4ee5\u5f88\u597d\u7684\u5b9e\u73b0\u670d\u52a1\u95f4\u7684\u89e3\u8026\u3001\u5f02\u6784\u5c4f\u853d\u548c\u4e92\u64cd\u4f5c\uff0c\u5e76\u63d0\u4f9b\u8def\u7531\u3001\u6d41\u63a7\u3001\u72b6\u6001\u7ba1\u7406\u3001\u76d1\u63a7\u7b49\u6cbb\u7406\u7279\u6027\u7684\u5171\u6027\u63d0\u70bc\u548c\u590d\u7528\uff0c\u589e\u5f3a\u6574\u4e2a\u67b6\u6784\u7684\u7075\u6d3b\u6027\u3001\u7ba1\u63a7\u80fd\u529b\u3001\u53ef\u6269\u5c55\u6027\u548c\u53ef\u7ef4\u62a4\u6027\u3002</p>\n<p>\u4f46\u76ee\u5149\u5f80\u4e0b\u4e00\u5c42\uff0c\u4f60\u4f1a\u53d1\u73b0\u5728\u4ece APP\u3001Service\uff0c\u5230\u540e\u53f0\u5f15\u64ce\u8fd9\u4e00\u5c42\u9762\uff0c\u5374\u8fd8\u662f\u5404\u4e2a\u5f15\u64ce\u5404\u81ea\u4e3a\u653f\uff0cClient-Server \u6a21\u5f0f\u7d27\u8026\u5408\u6ee1\u5929\u98de\u7684\u60c5\u51b5\u3002\u5728\u5927\u91cf\u7684\u4e0a\u5c42\u5e94\u7528\uff0c\u548c\u5927\u91cf\u7684\u5e95\u5c42\u5f15\u64ce\u4e4b\u95f4\uff0c\u7f3a\u4e4f\u4e00\u5c42\u901a\u7528\u7684\u201c\u4e2d\u95f4\u4ef6\u201d\u6846\u67b6\u8bbe\u8ba1\u3002\u7c7b\u4f3c\u4e0b\u56fe 2 \u7684\u7f51\u72b6\u3002</p>\n<p><img alt=\"\" src=\"https://imgconvert.csdnimg.cn/aHR0cHM6Ly91cGxvYWQtaW1hZ2VzLmppYW5zaHUuaW8vdXBsb2FkX2ltYWdlcy81NDI2NzctNzZmZDhkN2E0YmY1MzRjNQ?x-oss-process=image/format,png\"/></p>\n&lt;center&gt;\u56fe 2&lt;/center&gt;\n<p>\u8ba1\u7b97\u6cbb\u7406\uff0c\u5173\u6ce8\u7684\u6b63\u662f\u4e0a\u5c42\u5e94\u7528\u548c\u5e95\u5c42\u8ba1\u7b97\uff08\u5b58\u50a8\uff09\u5f15\u64ce\u4e4b\u95f4\uff0c\u4ece Client \u5230 Server \u7684\u8fde\u63a5\u5c42\u8303\u56f4\uff0c\u6240\u5b58\u5728\u7684\u7d27\u8026\u5408\u3001\u7075\u6d3b\u6027\u548c\u7ba1\u63a7\u80fd\u529b\u6b20\u7f3a\u3001\u7f3a\u4e4f\u590d\u7528\u80fd\u529b\u3001\u53ef\u6269\u5c55\u6027\u3001\u53ef\u7ef4\u62a4\u6027\u5dee\u7b49\u95ee\u9898\u3002\u8981\u8ba9\u590d\u6742\u5206\u5e03\u5f0f\u67b6\u6784\u73af\u5883\u4e0b\u5404\u79cd\u7c7b\u578b\u7684\u8ba1\u7b97\u4efb\u52a1\uff0c\u90fd\u80fd\u66f4\u7b80\u6d01\u3001\u7075\u6d3b\u3001\u6709\u5e8f\u3001\u53ef\u63a7\u7684\u63d0\u4ea4\u6267\u884c\uff0c\u548c\u6210\u529f\u8fd4\u56de\u7ed3\u679c\u3002\u5982\u4e0b\u56fe 3 \u6240\u793a\uff1a</p>\n<p><img alt=\"\" src=\"https://imgconvert.csdnimg.cn/aHR0cHM6Ly91cGxvYWQtaW1hZ2VzLmppYW5zaHUuaW8vdXBsb2FkX2ltYWdlcy81NDI2NzctYjQxYWUxNjQ3MGIwMTIyNA?x-oss-process=image/format,png\"/></p>\n&lt;center&gt;\u56fe 3&lt;/center&gt;\n<h3>\u8ba1\u7b97\u6cbb\u7406\u95ee\u9898\u63cf\u8ff0</h3>\n<p>\u66f4\u8be6\u7ec6\u7684\u6765\u770b\u8ba1\u7b97\u6cbb\u7406\u7684\u95ee\u9898\uff0c\u53ef\u4ee5\u5206\u4e3a\u5982\u4e0b\u6cbb\uff08 architecture\uff0c\u67b6\u6784\u5c42\u9762\uff09\u548c\u7406\uff08 insight\uff0c\u7ec6\u5316\u7279\u6027\uff09\u4e24\u4e2a\u5c42\u9762\u3002</p>\n<h4>\u8ba1\u7b97\u6cbb\u7406\u4e4b\u6cbb\uff08 architecture \uff09-\u67b6\u6784\u5c42\u9762\u95ee\u9898</h4>\n<p><strong>\u7d27\u8026\u5408\u95ee\u9898\uff0c\u4e0a\u5c42\u5e94\u7528\u548c\u5e95\u5c42\u8ba1\u7b97\u5b58\u50a8\u5f15\u64ce\u95f4\u7684 CS \u8fde\u63a5\u6a21\u5f0f\u3002</strong></p>\n<p>\u6240\u6709 APP &amp; Service \u548c\u5e95\u5c42\u8ba1\u7b97\u5b58\u50a8\u5f15\u64ce\uff0c\u90fd\u662f\u901a\u8fc7 Client-Server \u6a21\u5f0f\u76f8\u8fde\uff0c\u5904\u4e8e\u7d27\u8026\u5408\u72b6\u6001\u3002\u4ee5 Analytics Engine \u7684 Spark \u4e3a\u4f8b\uff0c\u5982\u4e0b\u56fe 4\uff1a</p>\n<p><img alt=\"\" src=\"https://imgconvert.csdnimg.cn/aHR0cHM6Ly91cGxvYWQtaW1hZ2VzLmppYW5zaHUuaW8vdXBsb2FkX2ltYWdlcy81NDI2NzctNWMzNzZiMDJjYWE1YTQ5MA?x-oss-process=image/format,png\"/></p>\n&lt;center&gt;\u56fe 4&lt;/center&gt;\n<p><strong>\u8fd9\u79cd\u72b6\u6001\u4f1a\u5e26\u6765\u5982\u4e0b\u95ee\u9898\uff1a</strong></p>\n<ul>\n<li>\n<p>\u5f15\u64ce client \u7684\u4efb\u4f55\u6539\u52a8\uff08\u5982\u7248\u672c\u5347\u7ea7\uff09\uff0c\u5c06\u76f4\u63a5\u5f71\u54cd\u6bcf\u4e00\u4e2a\u5d4c\u5165\u4e86\u8be5 client \u7684\u4e0a\u5c42\u5e94\u7528\uff1b\u5f53\u5e94\u7528\u7cfb\u7edf\u6570\u91cf\u4f17\u591a\u3001\u89c4\u6a21\u5e9e\u5927\u65f6\uff0c\u4e00\u6b21\u6539\u52a8\u7684\u6210\u672c\u4f1a\u5f88\u9ad8\u3002</p>\n</li>\n<li>\n<p>\u76f4\u8fde\u6a21\u5f0f\uff0c\u5bfc\u81f4\u4e0a\u5c42\u5e94\u7528\u7f3a\u4e4f\uff0c\u5bf9\u8de8\u5e95\u5c42\u8ba1\u7b97\u5b58\u50a8\u5f15\u64ce\u5b9e\u4f8b\u7ea7\u522b\u7684\uff0c\u8def\u7531\u9009\u62e9\u3001\u8d1f\u8f7d\u5747\u8861\u7b49\u80fd\u529b\uff1b\u6216\u8005\u8bf4\u4f9d\u8d56\u4e8e\u7279\u5b9a\u5e95\u5c42\u5f15\u64ce\u63d0\u4f9b\u7684\u7279\u5b9a\u8fde\u63a5\u65b9\u5f0f\u5b9e\u73b0\uff0c\u6709\u7684\u5f15\u64ce\u6709\u4e00\u4e9b\uff0c\u6709\u7684\u6ca1\u6709\u3002</p>\n</li>\n<li>\n<p>\u968f\u7740\u65f6\u95f4\u63a8\u79fb\uff0c\u4e0d\u65ad\u6709\u65b0\u7684\u4e0a\u5c42\u5e94\u7528\u548c\u65b0\u7684\u5e95\u5c42\u5f15\u64ce\u52a0\u5165\u8fdb\u6765\uff0c\u6574\u4f53\u67b6\u6784\u548c\u8c03\u7528\u5173\u7cfb\u5c06\u6108\u53d1\u590d\u6742\uff0c\u53ef\u6269\u5c55\u6027\u3001\u53ef\u9760\u6027\u548c\u53ef\u7ef4\u62a4\u6027\u964d\u4f4e\u3002</p>\n</li>\n</ul>\n<p>\u91cd\u590d\u9020\u8f6e\u5b50\u95ee\u9898\uff0c\u6bcf\u4e2a\u4e0a\u5c42\u5e94\u7528\u5de5\u5177\u7cfb\u7edf\u90fd\u8981\u91cd\u590d\u89e3\u51b3\u8ba1\u7b97\u6cbb\u7406\u95ee\u9898\u3002</p>\n<p>\u6bcf\u4e2a\u4e0a\u5c42\u5e94\u7528\u90fd\u8981\u91cd\u590d\u7684\u53bb\u96c6\u6210\u5404\u79cd client\uff0c\u521b\u5efa\u548c\u7ba1\u7406 client \u5230\u5f15\u64ce\u7684\u8fde\u63a5\u53ca\u5176\u72b6\u6001\uff0c\u5305\u62ec\u5e95\u5c42\u5f15\u64ce\u5143\u6570\u636e\u7684\u83b7\u53d6\u4e0e\u7ba1\u7406\u3002\u5728\u5e76\u53d1\u4f7f\u7528\u7684\u7528\u6237\u9010\u6e10\u53d8\u591a\u3001\u5e76\u53d1\u8ba1\u7b97\u4efb\u52a1\u91cf\u9010\u6e10\u53d8\u5927\u65f6\uff0c\u6bcf\u4e2a\u4e0a\u5c42\u5e94\u7528\u8fd8\u8981\u91cd\u590d\u7684\u53bb\u89e3\u51b3\u591a\u4e2a\u7528\u6237\u95f4\u5728 client \u7aef\u7684\u8d44\u6e90\u4e89\u7528\u3001\u6743\u9650\u9694\u79bb\uff0c\u8ba1\u7b97\u4efb\u52a1\u7684\u8d85\u65f6\u7ba1\u7406\u3001\u5931\u8d25\u91cd\u8bd5\u7b49\u7b49\u8ba1\u7b97\u6cbb\u7406\u95ee\u9898\u3002</p>\n<p><img alt=\"\" src=\"https://imgconvert.csdnimg.cn/aHR0cHM6Ly91cGxvYWQtaW1hZ2VzLmppYW5zaHUuaW8vdXBsb2FkX2ltYWdlcy81NDI2NzctNzAwYWI0OTFhMzI3ZDdiNQ?x-oss-process=image/format,png\"/></p>\n&lt;center&gt;\u56fe 5&lt;/center&gt;\n<p>\u60f3\u8c61\u4f60\u6709 10 \u4e2a\u5e76\u53d1\u4efb\u52a1\u6570\u8fc7\u767e\u7684\u4e0a\u5c42\u5e94\u7528\uff0c\u4e0d\u7ba1\u662f\u57fa\u4e8e Web \u7684 IDE \u5f00\u53d1\u73af\u5883\u3001\u53ef\u89c6\u5316 BI \u7cfb\u7edf\uff0c\u8fd8\u662f\u62a5\u8868\u7cfb\u7edf\u3001\u5de5\u4f5c\u6d41\u8c03\u5ea6\u7cfb\u7edf\u7b49\uff0c\u6bcf\u4e2a\u63a5\u5165 3 \u4e2a\u5e95\u5c42\u8ba1\u7b97\u5f15\u64ce\u3002\u4e0a\u8ff0\u7684\u8ba1\u7b97\u6cbb\u7406\u95ee\u9898\uff0c\u4f60\u53ef\u80fd\u5f97\u9010\u4e00\u91cd\u590d\u7684\u53bb\u89e3\u51b3 10*3=30 \u904d\uff0c\u800c\u8fd9\u6b63\u662f\u5f53\u524d\u5728\u5404\u4e2a\u516c\u53f8\u4e0d\u65ad\u53d1\u751f\u7684\u73b0\u5b9e\u60c5\u51b5\uff0c\u5176\u9020\u6210\u7684\u4eba\u529b\u6d6a\u8d39\u4e0d\u53ef\u5c0f\u89d1\u3002</p>\n<p><strong>\u6269\u5c55\u96be\u95ee\u9898\uff0c\u4e0a\u5c42\u5e94\u7528\u65b0\u589e\u5bf9\u63a5\u5e95\u5c42\u8ba1\u7b97\u5f15\u64ce\uff0c\u7ef4\u62a4\u6210\u672c\u9ad8\uff0c\u6539\u52a8\u5927\u3002</strong></p>\n<p>\u5728 CS \u7684\u7d27\u8026\u5408\u6a21\u5f0f\u4e0b\uff0c\u4e0a\u5c42\u5e94\u7528\u6bcf\u65b0\u589e\u5bf9\u63a5\u4e00\u4e2a\u5e95\u5c42\u8ba1\u7b97\u5f15\u64ce\uff0c\u90fd\u9700\u8981\u6709\u8f83\u5927\u6539\u52a8\u3002</p>\n<p>\u4ee5\u5bf9\u63a5 Spark \u4e3a\u4f8b\uff0c\u5728\u4e0a\u5c42\u5e94\u7528\u7cfb\u7edf\u4e2d\u7684\u6bcf\u4e00\u53f0\u9700\u8981\u63d0\u4ea4 Spark \u4f5c\u4e1a\u7684\u673a\u5668\uff0c\u90fd\u9700\u8981\u90e8\u7f72\u548c\u7ef4\u62a4\u597d Java \u548c Scala \u8fd0\u884c\u65f6\u73af\u5883\u548c\u53d8\u91cf\uff0c\u4e0b\u8f7d\u548c\u90e8\u7f72 Spark Client \u5305\uff0c\u4e14\u914d\u7f6e\u5e76\u7ef4\u62a4 Spark \u76f8\u5173\u7684\u73af\u5883\u53d8\u91cf\u3002\u5982\u679c\u8981\u4f7f\u7528 Spark on YARN \u6a21\u5f0f\uff0c\u90a3\u4e48\u4f60\u8fd8\u9700\u8981\u5728\u6bcf\u4e00\u53f0\u9700\u8981\u63d0\u4ea4 Spark \u4f5c\u4e1a\u7684\u673a\u5668\u4e0a\uff0c\u53bb\u90e8\u7f72\u548c\u7ef4\u62a4 Hadoop \u76f8\u5173\u7684 jar \u5305\u548c\u73af\u5883\u53d8\u91cf\u3002\u518d\u5982\u679c\u4f60\u7684 Hadoop \u96c6\u7fa4\u9700\u8981\u542f\u7528 Kerberos \u7684\uff0c\u90a3\u4e48\u5f88\u4e0d\u5e78\uff0c\u4f60\u8fd8\u9700\u8981\u5728\u4e0a\u8ff0\u7684\u6bcf\u53f0\u673a\u5668\u53bb\u7ef4\u62a4\u548c\u8c03\u8bd5 keytab\u3001principal \u7b49\u4e00\u5806 Kerberos \u76f8\u5173\u914d\u7f6e\u3002</p>\n<p><img alt=\"\" src=\"https://imgconvert.csdnimg.cn/aHR0cHM6Ly91cGxvYWQtaW1hZ2VzLmppYW5zaHUuaW8vdXBsb2FkX2ltYWdlcy81NDI2NzctZjQ1ZGI0M2YyZmM4ZTdhOQ?x-oss-process=image/format,png\"/></p>\n&lt;center&gt;\u56fe 6&lt;/center&gt;\n<p>\u8fd9\u8fd8\u4ec5\u4ec5\u662f\u5bf9\u63a5 Spark \u4e00\u4e2a\u5e95\u5c42\u5f15\u64ce\u3002\u968f\u7740\u4e0a\u5c42\u5e94\u7528\u7cfb\u7edf\u548c\u5e95\u5c42\u5f15\u64ce\u7684\u6570\u91cf\u589e\u591a\uff0c\u9700\u8981\u7ef4\u62a4\u7684\u5173\u7cfb\u4f1a\u662f\u4e2a\u7b1b\u5361\u5c14\u79ef\u5f0f\u7684\u589e\u957f\uff0c\u5149 Client \u548c\u914d\u7f6e\u7684\u90e8\u7f72\u7ef4\u62a4\uff0c\u5c31\u4f1a\u6210\u4e3a\u4e00\u4ef6\u5f88\u4ee4\u4eba\u5934\u75bc\u7684\u4e8b\u60c5\u3002</p>\n<p><strong>\u5e94\u7528\u5b64\u5c9b\u95ee\u9898\uff0c\u8de8\u4e0d\u540c\u5e94\u7528\u5de5\u5177\u3001\u4e0d\u540c\u8ba1\u7b97\u4efb\u52a1\u95f4\u7684\u4e92\u901a\u95ee\u9898\u3002</strong></p>\n<p>\u591a\u4e2a\u76f8\u4e92\u6709\u5173\u8054\u7684\u4e0a\u5c42\u5e94\u7528\uff0c\u5411\u540e\u53f0\u5f15\u64ce\u63d0\u4ea4\u6267\u884c\u7684\u4e0d\u540c\u8ba1\u7b97\u4efb\u52a1\u4e4b\u95f4\uff0c\u5f80\u5f80\u662f\u6709\u6240\u5173\u8054\u548c\u5171\u6027\u7684\uff0c\u6bd4\u5982\u9700\u8981\u5171\u4eab\u4e00\u4e9b\u7528\u6237\u5b9a\u4e49\u7684\u8fd0\u884c\u65f6\u73af\u5883\u53d8\u91cf\u3001\u51fd\u6570\u3001\u7a0b\u5e8f\u5305\u3001\u6570\u636e\u6587\u4ef6\u7b49\u3002\u5f53\u524d\u60c5\u51b5\u5f80\u5f80\u662f\u4e00\u4e2a\u4e2a\u5e94\u7528\u7cfb\u7edf\u5c31\u50cf\u4e00\u5ea7\u5ea7\u5b64\u5c9b\uff0c\u76f8\u5173\u4fe1\u606f\u548c\u8d44\u6e90\u65e0\u6cd5\u76f4\u63a5\u5171\u4eab\uff0c\u9700\u8981\u624b\u52a8\u5728\u4e0d\u540c\u5e94\u7528\u7cfb\u7edf\u91cc\u91cd\u590d\u5b9a\u4e49\u548c\u7ef4\u62a4\u3002</p>\n<p>\u5178\u578b\u4f8b\u5b50\u662f\u5728\u6570\u636e\u6279\u5904\u7406\u7a0b\u5e8f\u5f00\u53d1\u8fc7\u7a0b\u4e2d\uff0c\u7528\u6237\u5728\u6570\u636e\u63a2\u7d22\u5f00\u53d1 IDE \u7cfb\u7edf\u4e2d\u5b9a\u4e49\u7684\u4e00\u7cfb\u5217\u53d8\u91cf\u3001\u51fd\u6570\uff0c\u5230\u4e86\u6570\u636e\u53ef\u89c6\u5316\u7cfb\u7edf\u91cc\u5f80\u5f80\u53c8\u8981\u91cd\u65b0\u5b9a\u4e49\u4e00\u904d\uff1b IDE \u7cfb\u7edf\u8fd0\u884c\u751f\u6210\u7684\u6570\u636e\u6587\u4ef6\u4f4d\u7f6e\u548c\u540d\u79f0\uff0c\u4e0d\u80fd\u76f4\u63a5\u65b9\u4fbf\u7684\u4f20\u9012\u7ed9\u53ef\u89c6\u5316\u7cfb\u7edf\uff1b\u4f9d\u8d56\u7684\u7a0b\u5e8f\u5305\u4e5f\u9700\u8981\u4ece IDE \u7cfb\u7edf\u4e0b\u8f7d\u3001\u91cd\u65b0\u4e0a\u4f20\u5230\u53ef\u89c6\u5316\u7cfb\u7edf\uff1b\u5230\u4e86\u5de5\u4f5c\u6d41\u8c03\u5ea6\u7cfb\u7edf\uff0c\u8fd9\u4e2a\u8fc7\u7a0b\u8fd8\u8981\u518d\u91cd\u590d\u4e00\u904d\u3002\u4e0d\u540c\u4e0a\u5c42\u5e94\u7528\u95f4\uff0c\u8ba1\u7b97\u4efb\u52a1\u7684\u8fd0\u884c\u4f9d\u8d56\u7f3a\u4e4f\u4e92\u901a\u3001\u590d\u7528\u80fd\u529b\u3002</p>\n<p><img alt=\"\" src=\"https://imgconvert.csdnimg.cn/aHR0cHM6Ly91cGxvYWQtaW1hZ2VzLmppYW5zaHUuaW8vdXBsb2FkX2ltYWdlcy81NDI2NzctYTRkZjI0MzI0YjhlZjYxYQ?x-oss-process=image/format,png\"/></p>\n&lt;center&gt;\u56fe 7&lt;/center&gt;\n<h4>\u8ba1\u7b97\u6cbb\u7406\u4e4b\u7406\uff08 insight \uff09- \u7ec6\u5316\u7279\u6027\u95ee\u9898</h4>\n<p>\u9664\u4e86\u4e0a\u8ff0\u7684\u67b6\u6784\u5c42\u9762\u95ee\u9898\uff0c\u8981\u60f3\u8ba9\u590d\u6742\u5206\u5e03\u5f0f\u67b6\u6784\u73af\u5883\u4e0b\uff0c\u5404\u79cd\u7c7b\u578b\u7684\u8ba1\u7b97\u4efb\u52a1\uff0c\u90fd\u80fd\u66f4\u7b80\u6d01\u3001\u7075\u6d3b\u3001\u6709\u5e8f\u3001\u53ef\u63a7\u7684\u63d0\u4ea4\u6267\u884c\uff0c\u548c\u6210\u529f\u8fd4\u56de\u7ed3\u679c\uff0c\u8ba1\u7b97\u6cbb\u7406\u8fd8\u9700\u5173\u6ce8\u9ad8\u5e76\u53d1\uff0c\u9ad8\u53ef\u7528\uff0c\u591a\u79df\u6237\u9694\u79bb\uff0c\u8d44\u6e90\u7ba1\u63a7\uff0c\u5b89\u5168\u589e\u5f3a\uff0c\u8ba1\u7b97\u7b56\u7565\u7b49\u7b49\u7ec6\u5316\u7279\u6027\u95ee\u9898\u3002\u8fd9\u4e9b\u95ee\u9898\u90fd\u6bd4\u8f83\u76f4\u767d\u6613\u61c2\uff0c\u8fd9\u91cc\u5c31\u4e0d\u4e00\u4e00\u5c55\u5f00\u8bba\u8ff0\u4e86\u3002</p>\n<h2>\u57fa\u4e8e\u8ba1\u7b97\u4e2d\u95f4\u4ef6 Linkis \u7684\u8ba1\u7b97\u6cbb\u7406 - \u6cbb\u4e4b\u8def\uff08 Architecture \uff09</h2>\n<h3>Linkis \u67b6\u6784\u8bbe\u8ba1\u4ecb\u7ecd</h3>\n<h4>\u6838\u5fc3\u529f\u80fd\u6a21\u5757\u4e0e\u6d41\u7a0b</h4>\n<p>\u8ba1\u7b97\u4e2d\u95f4\u4ef6 Linkis\uff0c\u662f\u5fae\u4f17\u94f6\u884c\u4e13\u95e8\u8bbe\u8ba1\u7528\u6765\u89e3\u51b3\u4e0a\u8ff0\u7d27\u8026\u5408\u3001\u91cd\u590d\u9020\u8f6e\u5b50\u3001\u6269\u5c55\u96be\u3001\u5e94\u7528\u5b64\u5c9b\u7b49\u8ba1\u7b97\u6cbb\u7406\u95ee\u9898\u7684\u3002\u5f53\u524d\u4e3b\u8981\u89e3\u51b3\u7684\u662f\u590d\u6742\u5206\u5e03\u5f0f\u67b6\u6784\u7684\u5178\u578b\u573a\u666f-\u6570\u636e\u5e73\u53f0\u73af\u5883\u4e0b\u7684\u8ba1\u7b97\u6cbb\u7406\u95ee\u9898\u3002</p>\n<p>Linkis \u4f5c\u4e3a\u8ba1\u7b97\u4e2d\u95f4\u4ef6\uff0c\u5728\u4e0a\u5c42\u5e94\u7528\u548c\u5e95\u5c42\u5f15\u64ce\u4e4b\u95f4\uff0c\u6784\u5efa\u4e86\u4e00\u5c42\u4e2d\u95f4\u5c42\u3002\u80fd\u591f\u5e2e\u52a9\u4e0a\u5c42\u5e94\u7528\uff0c\u901a\u8fc7\u5176\u5bf9\u5916\u63d0\u4f9b\u7684\u6807\u51c6\u5316\u63a5\u53e3\uff08\u5982 HTTP, JDBC, Java \u2026\uff09\uff0c\u5feb\u901f\u7684\u8fde\u63a5\u5230\u591a\u79cd\u5e95\u5c42\u8ba1\u7b97\u5b58\u50a8\u5f15\u64ce\uff08\u5982 Spark\u3001Hive\u3001TiSpark\u3001MySQL\u3001Python \u7b49\uff09\uff0c\u63d0\u4ea4\u6267\u884c\u5404\u79cd\u7c7b\u578b\u7684\u8ba1\u7b97\u4efb\u52a1\uff0c\u5e76\u5b9e\u73b0\u8de8\u4e0a\u5c42\u5e94\u7528\u95f4\u7684\u8ba1\u7b97\u4efb\u52a1\u8fd0\u884c\u65f6\u4e0a\u4e0b\u6587\u548c\u4f9d\u8d56\u7684\u4e92\u901a\u548c\u5171\u4eab\u3002\u4e14\u901a\u8fc7\u63d0\u4f9b\u591a\u79df\u6237\u3001\u9ad8\u5e76\u53d1\u3001\u4efb\u52a1\u5206\u53d1\u548c\u7ba1\u7406\u7b56\u7565\u3001\u8d44\u6e90\u7ba1\u63a7\u7b49\u7279\u6027\u652f\u6301\uff0c\u4f7f\u5f97\u5404\u79cd\u8ba1\u7b97\u4efb\u52a1\u66f4\u7075\u6d3b\u3001\u53ef\u9760\u3001\u53ef\u63a7\u7684\u63d0\u4ea4\u6267\u884c\uff0c\u6210\u529f\u8fd4\u56de\u7ed3\u679c\uff0c\u5927\u5927\u964d\u4f4e\u4e86\u4e0a\u5c42\u5e94\u7528\u5728\u8ba1\u7b97\u6cbb\u7406\u5c42\u7684\u5f00\u53d1\u548c\u8fd0\u7ef4\u6210\u672c\u3001\u4e0e\u6574\u4e2a\u73af\u5883\u7684\u67b6\u6784\u590d\u6742\u5ea6\uff0c\u586b\u8865\u4e86\u901a\u7528\u8ba1\u7b97\u6cbb\u7406\u8f6f\u4ef6\u7684\u7a7a\u767d\u3002\uff08\u56fe 8\u30019 \uff09</p>\n<p><img alt=\"\" src=\"https://imgconvert.csdnimg.cn/aHR0cHM6Ly91cGxvYWQtaW1hZ2VzLmppYW5zaHUuaW8vdXBsb2FkX2ltYWdlcy81NDI2NzctNmQxNzc1OGE2MzQ3ZjU0OQ?x-oss-process=image/format,png\"/></p>\n&lt;center&gt;\u56fe 8&lt;/center&gt;\n<p><img alt=\"\" src=\"https://imgconvert.csdnimg.cn/aHR0cHM6Ly91cGxvYWQtaW1hZ2VzLmppYW5zaHUuaW8vdXBsb2FkX2ltYWdlcy81NDI2NzctZjQ1MjAyOWI4MGEyMmQ3YQ?x-oss-process=image/format,png\"/></p>\n&lt;center&gt;\u56fe 9&lt;/center&gt;\n<p>\u8981\u66f4\u8be6\u7ec6\u7684\u4e86\u89e3\u8ba1\u7b97\u4efb\u52a1\u901a\u8fc7 Linkis \u7684\u63d0\u4ea4\u6267\u884c\u8fc7\u7a0b\uff0c\u6211\u4eec\u5148\u6765\u770b\u770b Linkis \u6838\u5fc3\u7684\u201c\u8ba1\u7b97\u6cbb\u7406\u670d\u52a1\u201d\u90e8\u5206\u7684\u5185\u90e8\u67b6\u6784\u548c\u6d41\u7a0b\u3002\u5982\u4e0b\u56fe 10\uff1a</p>\n<p><img alt=\"\" src=\"https://imgconvert.csdnimg.cn/aHR0cHM6Ly91cGxvYWQtaW1hZ2VzLmppYW5zaHUuaW8vdXBsb2FkX2ltYWdlcy81NDI2NzctZjIyNjcxNGEzYmI1ZGQxYw?x-oss-process=image/format,png\"/></p>\n&lt;center&gt;\u56fe 10&lt;/center&gt;\n<p><strong>\u8ba1\u7b97\u6cbb\u7406\u670d\u52a1</strong>\uff1a\u8ba1\u7b97\u4e2d\u95f4\u4ef6\u7684\u6838\u5fc3\u8ba1\u7b97\u6846\u67b6\uff0c\u4e3b\u8981\u8d1f\u8d23\u4f5c\u4e1a\u8c03\u5ea6\u548c\u751f\u547d\u5468\u671f\u7ba1\u7406\u3001\u8ba1\u7b97\u8d44\u6e90\u7ba1\u7406\uff0c\u4ee5\u53ca\u5f15\u64ce\u8fde\u63a5\u5668\u7684\u751f\u547d\u5468\u671f\u7ba1\u7406\u3002</p>\n<p><strong>\u516c\u5171\u589e\u5f3a\u670d\u52a1</strong>\uff1a\u901a\u7528\u516c\u5171\u670d\u52a1\uff0c\u63d0\u4f9b\u57fa\u7840\u516c\u5171\u529f\u80fd\uff0c\u53ef\u670d\u52a1\u4e8e Linkis \u5404\u79cd\u670d\u52a1\u53ca\u4e0a\u5c42\u5e94\u7528\u7cfb\u7edf\u3002</p>\n<p><strong>\u5176\u4e2d\u8ba1\u7b97\u6cbb\u7406\u670d\u52a1\u7684\u4e3b\u8981\u6a21\u5757\u5982\u4e0b\uff1a</strong></p>\n<ul>\n<li>\n<p><strong>\u5165\u53e3\u670d\u52a1 Entrance</strong>\uff0c\u8d1f\u8d23\u63a5\u6536\u4f5c\u4e1a\u8bf7\u6c42\uff0c\u8f6c\u53d1\u4f5c\u4e1a\u8bf7\u6c42\u7ed9\u5bf9\u5e94\u7684 Engine\uff0c\u5e76\u5b9e\u73b0\u5f02\u6b65\u961f\u5217\u3001\u9ad8\u5e76\u53d1\u3001\u9ad8\u53ef\u7528\u3001\u591a\u79df\u6237\u9694\u79bb\u3002</p>\n</li>\n<li>\n<p><strong>\u5e94\u7528\u7ba1\u7406\u670d\u52a1 AppManager</strong>\uff0c\u8d1f\u8d23\u7ba1\u7406\u6240\u6709\u7684 EngineConnManager \u548c EngineConn\uff0c\u5e76\u63d0\u4f9b EngineConnManager \u7ea7\u548c EngineConn \u7ea7\u6807\u7b7e\u80fd\u529b\uff1b\u52a0\u8f7d\u65b0\u5f15\u64ce\u63d2\u4ef6\uff0c\u5411 RM \u7533\u8bf7\u8d44\u6e90\uff0c \u8981\u6c42 EM \u6839\u636e\u8d44\u6e90\u521b\u5efa EngineConn \uff1b\u57fa\u4e8e\u6807\u7b7e\u529f\u80fd\uff0c\u4e3a\u4f5c\u4e1a\u5206\u914d\u53ef\u7528 EngineConn\u3002</p>\n</li>\n<li>\n<p><strong>\u8d44\u6e90\u7ba1\u7406\u670d\u52a1 ResourceManager</strong>\uff0c\u63a5\u6536\u8d44\u6e90\u7533\u8bf7\uff0c\u5206\u914d\u8d44\u6e90\uff0c\u63d0\u4f9b\u7cfb\u7edf\u7ea7\u3001\u7528\u6237\u7ea7\u8d44\u6e90\u7ba1\u63a7\u80fd\u529b\uff0c\u5e76\u4e3a EngineConnManager \u7ea7\u548c EngineConn \u63d0\u4f9b\u8d1f\u8f7d\u7ba1\u63a7\u3002</p>\n</li>\n<li>\n<p><strong>\u5f15\u64ce\u8fde\u63a5\u5668\u7ba1\u7406\u670d\u52a1 EngineConn Manager</strong>\uff0c\u8d1f\u8d23\u542f\u52a8 EngineConn\uff0c\u7ba1\u7406 EngineConn \u7684\u751f\u547d\u5468\u671f\uff0c\u5e76\u5b9a\u65f6\u5411 RM \u4e0a\u62a5\u8d44\u6e90\u548c\u8d1f\u8f7d\u60c5\u51b5\u3002</p>\n</li>\n<li>\n<p><strong>\u5f15\u64ce\u8fde\u63a5\u5668 EngineConn</strong>\uff0c\u8d1f\u8d23\u4e0e\u5e95\u5c42\u5f15\u64ce\u4ea4\u4e92\uff0c\u89e3\u6790\u548c\u8f6c\u6362\u7528\u6237\u4f5c\u4e1a\uff0c\u63d0\u4ea4\u8ba1\u7b97\u4efb\u52a1\u7ed9\u5e95\u5c42\u5f15\u64ce\uff0c\u5e76\u5b9e\u65f6\u76d1\u542c\u5e95\u5c42\u5f15\u64ce\u6267\u884c\u60c5\u51b5\uff0c\u56de\u63a8\u76f8\u5173\u65e5\u5fd7\u3001\u8fdb\u5ea6\u548c\u72b6\u6001\u7ed9 Entrance\u3002</p>\n</li>\n</ul>\n<p>\u5982\u56fe 10 \u6240\u793a\uff0c\u4e00\u4e2a\u4f5c\u4e1a\u7684\u63d0\u4ea4\u6267\u884c\u4e3b\u8981\u5206\u4e3a\u4ee5\u4e0b 11 \u6b65\uff1a</p>\n<p>1. \u4e0a\u5c42\u5e94\u7528\u5411\u8ba1\u7b97\u4e2d\u95f4\u4ef6\u63d0\u4ea4\u4f5c\u4e1a\uff0c\u5fae\u670d\u52a1\u7f51\u5173 SpringCloud Gateway \u63a5\u6536\u4f5c\u4e1a\u5e76\u8f6c\u53d1\u7ed9 Entrance\u3002</p>\n<p>2. Entrance \u6d88\u8d39\u4f5c\u4e1a\uff0c\u4e3a\u4f5c\u4e1a\u5411 AppManager \u7533\u8bf7\u53ef\u7528 EngineConn\u3002</p>\n<p>3. \u5982\u679c\u4e0d\u5b58\u5728\u53ef\u590d\u7528\u7684 Engine\uff0cAppManager \u5c1d\u8bd5\u5411 ResourceManager \u7533\u8bf7\u8d44\u6e90\uff0c\u4e3a\u4f5c\u4e1a\u542f\u52a8\u4e00\u4e2a\u65b0 EngineConn\u3002</p>\n<ol>\n<li>\u7533\u8bf7\u5230\u8d44\u6e90\uff0c\u8981\u6c42 EngineConnManager \u4f9d\u7167\u8d44\u6e90\u542f\u52a8\u65b0 EngineConn</li>\n</ol>\n<p>5. EngineConnManager \u542f\u52a8\u65b0 EngineConn\uff0c\u5e76\u4e3b\u52a8\u56de\u63a8\u65b0 EngineConn \u4fe1\u606f\u3002</p>\n<p>6. AppManager \u5c06\u65b0 EngineConn \u5206\u914d\u7ed9 Entrance\uff0cEntrance \u5c06 EngineConn \u5206\u914d\u7ed9\u7528\u6237\u4f5c\u4e1a\uff0c\u4f5c\u4e1a\u5f00\u59cb\u6267\u884c\uff0c\u5c06\u8ba1\u7b97\u4efb\u52a1\u63d0\u4ea4\u7ed9 EngineConn\u3002</p>\n<p>7. EngineConn \u5c06\u8ba1\u7b97\u4efb\u52a1\u63d0\u4ea4\u7ed9\u5e95\u5c42\u8ba1\u7b97\u5f15\u64ce\u3002</p>\n<p>8. EngineConn \u5b9e\u65f6\u76d1\u542c\u5e95\u5c42\u5f15\u64ce\u6267\u884c\u60c5\u51b5\uff0c\u56de\u63a8\u76f8\u5173\u65e5\u5fd7\u3001\u8fdb\u5ea6\u548c\u72b6\u6001\u7ed9 Entrance\uff0cEntrance \u901a\u8fc7 WebSocket\uff0c\u4e3b\u52a8\u56de\u63a8 EngineConn \u4f20\u8fc7\u6765\u7684\u65e5\u5fd7\u3001\u8fdb\u5ea6\u548c\u72b6\u6001\u7ed9\u4e0a\u5c42\u5e94\u7528\u7cfb\u7edf\u3002</p>\n<p>9. EngineConn \u6267\u884c\u5b8c\u6210\u540e\uff0c\u56de\u63a8\u8ba1\u7b97\u4efb\u52a1\u7684\u72b6\u6001\u548c\u7ed3\u679c\u96c6\u4fe1\u606f\uff0cEntrance \u5c06\u4f5c\u4e1a\u548c\u7ed3\u679c\u96c6\u4fe1\u606f\u66f4\u65b0\u5230 JobHistory\uff0c\u5e76\u901a\u77e5\u4e0a\u5c42\u5e94\u7528\u7cfb\u7edf\u3002</p>\n<p>10. \u4e0a\u5c42\u5e94\u7528\u7cfb\u7edf\u8bbf\u95ee JobHistory\uff0c\u62ff\u5230\u4f5c\u4e1a\u548c\u7ed3\u679c\u96c6\u4fe1\u606f\u3002</p>\n<p>11. \u4e0a\u5c42\u5e94\u7528\u7cfb\u7edf\u8bbf\u95ee Storage\uff0c\u8bf7\u6c42\u4f5c\u4e1a\u7ed3\u679c\u96c6\u3002</p>\n<h4>\u8ba1\u7b97\u4efb\u52a1\u7ba1\u7406\u7b56\u7565\u652f\u6301</h4>\n<p>\u5728\u590d\u6742\u5206\u5e03\u5f0f\u73af\u5883\u4e0b\uff0c\u4e00\u4e2a\u8ba1\u7b97\u4efb\u52a1\u5f80\u5f80\u4e0d\u5355\u4f1a\u662f\u7b80\u5355\u7684\u63d0\u4ea4\u6267\u884c\u548c\u8fd4\u56de\u7ed3\u679c\uff0c\u8fd8\u53ef\u80fd\u9700\u8981\u9762\u5bf9\u63d0\u4ea4\u5931\u8d25\u3001\u6267\u884c\u5931\u8d25\u3001hang \u4f4f\u7b49\u95ee\u9898\uff0c\u4e14\u5728\u5927\u91cf\u5e76\u53d1\u573a\u666f\u4e0b\u8fd8\u9700\u901a\u8fc7\u8ba1\u7b97\u4efb\u52a1\u7684\u8c03\u5ea6\u5206\u53d1\uff0c\u89e3\u51b3\u79df\u6237\u95f4\u4e92\u76f8\u5f71\u54cd\u3001\u8d1f\u8f7d\u5747\u8861\u7b49\u95ee\u9898\u3002</p>\n<p>Linkis \u901a\u8fc7\u5bf9\u8ba1\u7b97\u4efb\u52a1\u7684\u6807\u7b7e\u5316\uff0c\u5b9e\u73b0\u4e86\u5728\u4efb\u52a1\u8c03\u5ea6\u3001\u5206\u53d1\u3001\u8def\u7531\u7b49\u65b9\u9762\u8ba1\u7b97\u4efb\u52a1\u7ba1\u7406\u7b56\u7565\u7684\u652f\u6301\uff0c\u5e76\u53ef\u6309\u9700\u914d\u7f6e\u8d85\u65f6\u3001\u81ea\u52a8\u91cd\u8bd5\uff0c\u53ca\u7070\u5ea6\u3001\u591a\u6d3b\u7b49\u7b56\u7565\u652f\u6301\u3002\u5982\u4e0b\u56fe 11\u3002</p>\n<p><img alt=\"\" src=\"https://imgconvert.csdnimg.cn/aHR0cHM6Ly91cGxvYWQtaW1hZ2VzLmppYW5zaHUuaW8vdXBsb2FkX2ltYWdlcy81NDI2NzctZTEyYTMwNDY1NDNmZDQzNg?x-oss-process=image/format,png\"/></p>\n&lt;center&gt;\u56fe 11&lt;/center&gt;\n<h4>\u57fa\u4e8e Spring Cloud \u5fae\u670d\u52a1\u6846\u67b6</h4>\n<p>\u8bf4\u5b8c\u4e86\u4e1a\u52a1\u67b6\u6784\uff0c\u6211\u4eec\u73b0\u5728\u6765\u804a\u804a\u6280\u672f\u67b6\u6784\u3002\u5728\u8ba1\u7b97\u6cbb\u7406\u5c42\u73af\u5883\u4e0b\uff0c\u5f88\u591a\u7c7b\u578b\u7684\u8ba1\u7b97\u4efb\u52a1\u5177\u6709\u751f\u547d\u5468\u671f\u8f83\u77ed\u7684\u7279\u5f81\uff0c\u5982\u4e00\u4e2a Spark job \u53ef\u80fd\u51e0\u5341\u79d2\u5230\u51e0\u5206\u949f\u5c31\u6267\u884c\u5b8c\uff0cEngineConn \uff08 EnginConnector \uff09\u4f1a\u662f\u5927\u91cf\u52a8\u6001\u542f\u505c\u7684\u72b6\u6001\u3002\u524d\u7aef\u7528\u6237\u548c Linkis \u4e2d\u5176\u4ed6\u7ba1\u7406\u89d2\u8272\u7684\u670d\u52a1\uff0c\u9700\u8981\u80fd\u591f\u53ca\u65f6\u52a8\u6001\u53d1\u73b0\u76f8\u5173\u670d\u52a1\u5b9e\u4f8b\u7684\u72b6\u6001\u53d8\u5316\uff0c\u5e76\u83b7\u53d6\u6700\u65b0\u7684\u670d\u52a1\u5b9e\u4f8b\u8bbf\u95ee\u5730\u5740\u4fe1\u606f\u3002\u540c\u65f6\u9700\u8981\u8003\u8651\uff0c\u5404\u6a21\u5757\u95f4\u7684\u901a\u4fe1\u3001\u8def\u7531\u3001\u534f\u8c03\uff0c\u53ca\u5404\u6a21\u5757\u7684\u6a2a\u5411\u6269\u5c55\u3001\u8d1f\u8f7d\u5747\u8861\u3001\u9ad8\u53ef\u7528\u7b49\u80fd\u529b\u3002</p>\n<p>\u57fa\u4e8e\u4ee5\u4e0a\u9700\u6c42\uff0cLinkis \u5b9e\u9645\u662f\u57fa\u4e8e Spring Cloud \u5fae\u670d\u52a1\u6846\u67b6\u6280\u672f\uff0c\u5c06\u4e0a\u8ff0\u7684\u6bcf\u4e00\u4e2a\u6a21\u5757 /\u89d2\u8272\uff0c\u90fd\u5c01\u88c5\u6210\u4e86\u4e00\u4e2a\u5fae\u670d\u52a1\uff0c\u6784\u5efa\u4e86\u591a\u4e2a\u5fae\u670d\u52a1\u7ec4\uff0c\u6574\u5408\u5f62\u6210\u4e86 Linkis \u7684\u5b8c\u6574\u8ba1\u7b97\u4e2d\u95f4\u4ef6\u80fd\u529b\u3002\u5982\u4e0b\u56fe 12\uff1a</p>\n<p><img alt=\"\" src=\"https://imgconvert.csdnimg.cn/aHR0cHM6Ly91cGxvYWQtaW1hZ2VzLmppYW5zaHUuaW8vdXBsb2FkX2ltYWdlcy81NDI2NzctNGM0YTRiMmMyZThmZDI2YQ?x-oss-process=image/format,png\"/></p>\n&lt;center&gt;\u56fe 12&lt;/center&gt;\n<p>\u4ece\u591a\u79df\u6237\u7ba1\u7406\u89d2\u5ea6\uff0c\u4e0a\u8ff0\u670d\u52a1\u53ef\u533a\u5206\u4e3a\u79df\u6237\u76f8\u5173\u670d\u52a1\uff0c\u548c\u79df\u6237\u65e0\u5173\u670d\u52a1\u4e24\u79cd\u7c7b\u578b\u3002\u79df\u6237\u76f8\u5173\u670d\u52a1\uff0c\u662f\u6307\u4e00\u4e9b\u4efb\u52a1\u903b\u8f91\u5904\u7406\u8d1f\u8377\u91cd\u3001\u8d44\u6e90\u6d88\u8017\u9ad8\uff0c\u6216\u9700\u8981\u6839\u636e\u5177\u4f53\u79df\u6237\u3001\u7528\u6237\u3001\u7269\u7406\u673a\u5668\u7b49\uff0c\u505a\u9694\u79bb\u5212\u5206\u3001\u907f\u514d\u76f8\u4e92\u5f71\u54cd\u7684\u670d\u52a1\uff0c\u5982 Entrance\u3001EnginConn \uff08 EnginConnector \uff09 Manager\u3001EnginConn \uff1b\u5176\u4ed6\u5982 App Manger\u3001Resource Manager\u3001Context Service \u7b49\u670d\u52a1\uff0c\u90fd\u662f\u79df\u6237\u65e0\u5173\u7684\u3002</p>\n<p><strong>Eureka</strong> \u627f\u62c5\u4e86\u5fae\u670d\u52a1\u52a8\u6001\u6ce8\u518c\u4e0e\u53d1\u73b0\u4e2d\u5fc3\uff0c\u53ca\u6240\u6709\u79df\u6237\u65e0\u5173\u670d\u52a1\u7684\u8d1f\u8f7d\u5747\u8861\u3001\u6545\u969c\u8f6c\u79fb\u529f\u80fd\u3002</p>\n<p>Eureka \u6709\u4e2a\u5c40\u9650\uff0c\u5c31\u662f\u5728\u5176\u5ba2\u6237\u7aef\uff0c\u5bf9\u540e\u7aef\u5fae\u670d\u52a1\u5b9e\u4f8b\u7684\u53d1\u73b0\u4e0e\u72b6\u6001\u5237\u65b0\u673a\u5236\uff0c\u662f\u5ba2\u6237\u7aef\u4e3b\u52a8\u8f6e\u8be2\u5237\u65b0\uff0c\u6700\u5feb\u53ef\u8bbe 1 \u79d2 1 \u6b21\uff08\u5b9e\u9645\u8981\u51e0\u79d2\u624d\u80fd\u5b8c\u6210\u5237\u65b0\uff09\u3002\u8fd9\u6837\u5728 Linkis \u8fd9\u79cd\u9700\u8981\u5feb\u901f\u5237\u65b0\u5927\u91cf\u540e\u7aef EnginConn \u7b49\u670d\u52a1\u7684\u72b6\u6001\u7684\u573a\u666f\u4e0b\uff0c\u65f6\u6548\u5f97\u4e0d\u5230\u6ee1\u8db3\uff0c\u4e14\u5b9a\u65f6\u8f6e\u8be2\u5237\u65b0\u5bf9 Eureka server\u3001\u5bf9\u540e\u7aef\u5fae\u670d\u52a1\u5b9e\u4f8b\u7684\u6210\u672c\u90fd\u5f88\u9ad8\u3002</p>\n<p>\u4e3a\u6b64\u6211\u4eec\u5bf9 Spring Cloud Ribbon \u505a\u4e86\u6539\u9020\uff0c\u5728\u5176\u4e2d\u5c01\u88c5\u4e86 Eureka client \u7684\u5fae\u670d\u52a1\u5b9e\u4f8b\u72b6\u6001\u5237\u65b0\u65b9\u6cd5\uff0c\u5e76\u628a\u5b83\u505a\u6210\u6ee1\u8db3\u6761\u4ef6\u4e3b\u52a8\u8bf7\u6c42\u5237\u65b0\uff0c\u800c\u4e0d\u4f1a\u518d\u9891\u7e41\u7684\u5b9a\u671f\u8f6e\u8be2\u3002\u4ece\u800c\u5728\u6ee1\u8db3\u65f6\u6548\u7684\u540c\u65f6\uff0c\u5927\u5927\u964d\u4f4e\u4e86\u72b6\u6001\u83b7\u53d6\u7684\u6210\u672c\u3002\u5982\u4e0b\u56fe 13\uff1a</p>\n<p><img alt=\"\" src=\"https://imgconvert.csdnimg.cn/aHR0cHM6Ly91cGxvYWQtaW1hZ2VzLmppYW5zaHUuaW8vdXBsb2FkX2ltYWdlcy81NDI2NzctZTg0OWNhN2U3NTljNzQyYg?x-oss-process=image/format,png\"/></p>\n&lt;center&gt;\u56fe 13&lt;/center&gt;\n<p><strong>Spring Cloud Gateway</strong> \u627f\u62c5\u4e86\u5916\u90e8\u8bf7\u6c42 Linkis \u7684\u5165\u53e3\u7f51\u5173\u7684\u89d2\u8272\uff0c\u5e2e\u52a9\u5728\u670d\u52a1\u5b9e\u4f8b\u4e0d\u65ad\u53d1\u751f\u53d8\u5316\u7684\u60c5\u51b5\u4e0b\uff0c\u7b80\u5316\u524d\u7aef\u7528\u6237\u7684\u8c03\u7528\u903b\u8f91\uff0c\u5feb\u901f\u65b9\u4fbf\u7684\u83b7\u53d6\u6700\u65b0\u7684\u670d\u52a1\u5b9e\u4f8b\u8bbf\u95ee\u5730\u5740\u4fe1\u606f\u3002</p>\n<p>Spring Cloud Gateway \u6709\u4e2a\u5c40\u9650\uff0c\u5c31\u662f\u4e00\u4e2a WebSocket \u5ba2\u6237\u7aef\u53ea\u80fd\u5c06\u8bf7\u6c42\u8f6c\u53d1\u7ed9\u4e00\u4e2a\u7279\u5b9a\u7684\u540e\u53f0\u670d\u52a1\uff0c\u65e0\u6cd5\u5b8c\u6210\u4e00\u4e2a WebSocket \u5ba2\u6237\u7aef\u901a\u8fc7\u7f51\u5173 API \u5bf9\u63a5\u540e\u53f0\u591a\u4e2a WebSocket \u5fae\u670d\u52a1\uff0c\u800c\u8fd9\u5728\u6211\u4eec\u7684 Entrance HA \u7b49\u573a\u666f\u9700\u8981\u7528\u5230\u3002</p>\n<p>\u4e3a\u6b64 Linkis \u5bf9 Spring Cloud Gateway \u505a\u4e86\u76f8\u5e94\u6539\u9020\uff0c\u5728 Gateway \u4e2d\u5b9e\u73b0\u4e86 WebSocket \u8def\u7531\u8f6c\u53d1\u5668\uff0c\u7528\u4e8e\u4e0e\u5ba2\u6237\u7aef\u5efa\u7acb WebSocket \u8fde\u63a5\u3002\u5efa\u7acb\u8fde\u63a5\u6210\u529f\u540e\uff0c\u4f1a\u81ea\u52a8\u5206\u6790\u5ba2\u6237\u7aef\u7684 WebSocket \u8bf7\u6c42\uff0c\u901a\u8fc7\u89c4\u5219\u5224\u65ad\u51fa\u8bf7\u6c42\u8be5\u8f6c\u53d1\u7ed9\u54ea\u4e2a\u540e\u7aef\u5fae\u670d\u52a1\uff0c\u7136\u540e\u5c06 WebSocket \u8bf7\u6c42\u8f6c\u53d1\u7ed9\u5bf9\u5e94\u7684\u540e\u7aef\u5fae\u670d\u52a1\u5b9e\u4f8b\u3002\u8be6\u89c1 Github \u4e0a Linkis \u7684 Wiki \u4e2d\uff0c\u201cGateway \u7684\u591a WebSocket \u8bf7\u6c42\u8f6c\u53d1\u5b9e\u73b0\u201d\u4e00\u6587\u3002</p>\n<p><img alt=\"\" src=\"https://imgconvert.csdnimg.cn/aHR0cHM6Ly91cGxvYWQtaW1hZ2VzLmppYW5zaHUuaW8vdXBsb2FkX2ltYWdlcy81NDI2NzctNTE5NDc3MTgwN2Q5ZWJhYg?x-oss-process=image/format,png\"/></p>\n&lt;center&gt;\u56fe 14&lt;/center&gt;\n<p>**Spring Cloud OpenFeign ** \u63d0\u4f9b\u7684 HTTP \u8bf7\u6c42\u8c03\u7528\u63a5\u53e3\u5316\u3001\u89e3\u6790\u6a21\u677f\u5316\u80fd\u529b\uff0c\u5e2e\u52a9 Linkis \u6784\u5efa\u4e86\u5e95\u5c42 RPC \u901a\u4fe1\u6846\u67b6\u3002\u4f46\u57fa\u4e8e Feign \u7684\u5fae\u670d\u52a1\u4e4b\u95f4 HTTP \u63a5\u53e3\u7684\u8c03\u7528\uff0c\u53ea\u80fd\u6ee1\u8db3\u7b80\u5355\u7684 A \u5fae\u670d\u52a1\u5b9e\u4f8b\u6839\u636e\u7b80\u5355\u7684\u89c4\u5219\u968f\u673a\u9009\u62e9 B \u5fae\u670d\u52a1\u4e4b\u4e2d\u7684\u67d0\u4e2a\u670d\u52a1\u5b9e\u4f8b\uff0c\u800c\u8fd9\u4e2a B \u5fae\u670d\u52a1\u5b9e\u4f8b\u5982\u679c\u60f3\u5f02\u6b65\u56de\u4f20\u4fe1\u606f\u7ed9\u8c03\u7528\u65b9\uff0c\u662f\u65e0\u6cd5\u5b9e\u73b0\u7684\u3002\u540c\u65f6\uff0c\u7531\u4e8e Feign \u53ea\u652f\u6301\u7b80\u5355\u7684\u670d\u52a1\u9009\u53d6\u89c4\u5219\uff0c\u65e0\u6cd5\u505a\u5230\u5c06\u8bf7\u6c42\u8f6c\u53d1\u7ed9\u6307\u5b9a\u7684\u5fae\u670d\u52a1\u5b9e\u4f8b\uff0c\u65e0\u6cd5\u505a\u5230\u5c06\u4e00\u4e2a\u8bf7\u6c42\u5e7f\u64ad\u7ed9\u63a5\u6536\u65b9\u5fae\u670d\u52a1\u7684\u6240\u6709\u5b9e\u4f8b\u3002Linkis \u57fa\u4e8e Feign \u5b9e\u73b0\u4e86\u4e00\u5957\u81ea\u5df1\u7684\u5e95\u5c42 RPC \u901a\u4fe1\u65b9\u6848\uff0c\u96c6\u6210\u5230\u4e86\u6240\u6709 Linkis \u7684\u5fae\u670d\u52a1\u4e4b\u4e2d\u3002\u4e00\u4e2a\u5fae\u670d\u52a1\u65e2\u53ef\u4ee5\u4f5c\u4e3a\u8bf7\u6c42\u8c03\u7528\u65b9\uff0c\u4e5f\u53ef\u4ee5\u4f5c\u4e3a\u8bf7\u6c42\u63a5\u6536\u65b9\u3002\u4f5c\u4e3a\u8bf7\u6c42\u8c03\u7528\u65b9\u65f6\uff0c\u5c06\u901a\u8fc7 Sender \u8bf7\u6c42\u76ee\u6807\u63a5\u6536\u65b9\u5fae\u670d\u52a1\u7684 Receiver \uff1b\u4f5c\u4e3a\u8bf7\u6c42\u63a5\u6536\u65b9\u65f6\uff0c\u5c06\u63d0\u4f9b Receiver \u7528\u6765\u5904\u7406\u8bf7\u6c42\u63a5\u6536\u65b9 Sender \u53d1\u9001\u8fc7\u6765\u7684\u8bf7\u6c42\uff0c\u4ee5\u4fbf\u5b8c\u6210\u540c\u6b65\u54cd\u5e94\u6216\u5f02\u6b65\u54cd\u5e94\u3002\u5982\u4e0b\u56fe\u793a\u610f\u3002\u8be6\u89c1 GitHub \u4e0a Linkis \u7684 Wiki \u4e2d\uff0c\u201cLinkis RPC \u67b6\u6784\u4ecb\u7ecd\u201d\u4e00\u6587\u3002</p>\n<p><img alt=\"\" src=\"https://imgconvert.csdnimg.cn/aHR0cHM6Ly91cGxvYWQtaW1hZ2VzLmppYW5zaHUuaW8vdXBsb2FkX2ltYWdlcy81NDI2NzctOGM4MTRkODU1NTk1ZjZhMw?x-oss-process=image/format,png\"/></p>\n&lt;center&gt;\u56fe 15&lt;/center&gt;\n<p>\u81f3\u6b64\uff0cLinkis \u5bf9\u4e0a\u5c42\u5e94\u7528\u548c\u5e95\u5c42\u5f15\u64ce\u7684\u89e3\u8026\u539f\u7406\uff0c\u5176\u6838\u5fc3\u67b6\u6784\u4e0e\u6d41\u7a0b\u8bbe\u8ba1\uff0c\u53ca\u57fa\u4e8e Spring Cloud \u5fae\u670d\u52a1\u6846\u67b6\u5b9e\u73b0\u7684\uff0c\u5404\u6a21\u5757\u5fae\u670d\u52a1\u5316\u52a8\u6001\u7ba1\u7406\u3001\u901a\u4fe1\u8def\u7531\u3001\u6a2a\u5411\u6269\u5c55\u80fd\u529b\u4ecb\u7ecd\u5b8c\u6bd5\u3002</p>\n<h3>\u89e3\u8026\uff1aLinkis \u5982\u4f55\u89e3\u8026\u4e0a\u5c42\u5e94\u7528\u548c\u5e95\u5c42\u5f15\u64ce</h3>\n<p>Linkis \u4f5c\u4e3a\u8ba1\u7b97\u4e2d\u95f4\u4ef6\uff0c\u5728\u4e0a\u5c42\u5e94\u7528\u548c\u5e95\u5c42\u5f15\u64ce\u4e4b\u95f4\uff0c\u6784\u5efa\u4e86\u4e00\u5c42\u4e2d\u95f4\u5c42\u3002\u4e0a\u5c42\u5e94\u7528\u6240\u6709\u8ba1\u7b97\u4efb\u52a1\uff0c\u5148\u901a\u8fc7 HTTP\u3001WebSocket\u3001Java \u7b49\u63a5\u53e3\u65b9\u5f0f\u63d0\u4ea4\u7ed9 Linkis\uff0c\u518d\u7531 Linkis \u8f6c\u4ea4\u7ed9\u5e95\u5c42\u5f15\u64ce\u3002\u539f\u6709\u7684\u4e0a\u5c42\u5e94\u7528\u4ee5 CS \u6a21\u5f0f\u76f4\u8fde\u5e95\u5c42\u5f15\u64ce\u7684\u7d27\u8026\u5408\u5f97\u4ee5\u89e3\u9664\uff0c\u56e0\u6b64\u5b9e\u73b0\u4e86\u89e3\u8026\u3002\u5982\u4e0b\u56fe 16 \u6240\u793a\uff1a</p>\n<p><img alt=\"\" src=\"https://imgconvert.csdnimg.cn/aHR0cHM6Ly91cGxvYWQtaW1hZ2VzLmppYW5zaHUuaW8vdXBsb2FkX2ltYWdlcy81NDI2NzctNTRmNTlmMGM1NzlkOWIwMA?x-oss-process=image/format,png\"/></p>\n&lt;center&gt;\u56fe 16&lt;/center&gt;\n<p>\u901a\u8fc7\u89e3\u8026\uff0c\u5e95\u5c42\u5f15\u64ce\u7684\u53d8\u52a8\u6709\u4e86 Linkis \u8fd9\u5c42\u4e2d\u95f4\u4ef6\u7f13\u51b2\uff0c\u5982\u5f15\u64ce client \u7684\u7248\u672c\u5347\u7ea7\uff0c\u65e0\u9700\u518d\u5bf9\u6bcf\u4e00\u4e2a\u5bf9\u63a5\u7684\u4e0a\u5c42\u5e94\u7528\u505a\u9010\u4e2a\u6539\u52a8\uff0c\u53ef\u5728 Linkis \u5c42\u7edf\u4e00\u5b8c\u6210\u3002\u5e76\u80fd\u5728 Linkis \u5c42\uff0c\u5b9e\u73b0\u5bf9\u4e0a\u5c42\u5e94\u7528\u66f4\u52a0\u900f\u660e\u548c\u53cb\u597d\u7684\u5347\u7ea7\u7b56\u7565\uff0c\u5982\u7070\u5ea6\u5207\u6362\u3001\u591a\u6d3b\u7b49\u7b56\u7565\u652f\u6301\u3002\u4e14\u5373\u4f7f\u540e\u7ee7\u63a5\u5165\u66f4\u591a\u4e0a\u5c42\u5e94\u7528\u548c\u5e95\u5c42\u5f15\u64ce\uff0c\u6574\u4e2a\u73af\u5883\u590d\u6742\u5ea6\u4e5f\u4e0d\u4f1a\u6709\u5927\u7684\u53d8\u5316\uff0c\u5927\u5927\u964d\u4f4e\u4e86\u5f00\u53d1\u8fd0\u7ef4\u5de5\u4f5c\u8d1f\u62c5\u3002</p>\n<h3>\u590d\u7528\uff1a\u5bf9\u4e8e\u4e0a\u5c42\u5e94\u7528\uff0cLinkis \u5982\u4f55\u51dd\u7ec3\u8ba1\u7b97\u6cbb\u7406\u6a21\u5757\u4f9b\u590d\u7528\uff0c\u907f\u514d\u91cd\u590d\u5f00\u53d1</h3>\n<h4>\u4e0a\u5c42\u5e94\u7528\u590d\u7528 Linkis \u793a\u4f8b\uff08 Scriptis \uff09</h4>\n<p>\u6709\u4e86 Linkis\uff0c\u4e0a\u5c42\u5e94\u7528\u53ef\u4ee5\u57fa\u4e8e Linkis\uff0c\u5feb\u901f\u5b9e\u73b0\u5bf9\u591a\u79cd\u540e\u53f0\u8ba1\u7b97\u5b58\u50a8\u5f15\u64ce\u7684\u5bf9\u63a5\u652f\u6301\uff0c\u53ca\u53d8\u91cf\u3001\u51fd\u6570\u7b49\u81ea\u5b9a\u4e49\u4e0e\u7ba1\u7406\u3001\u8d44\u6e90\u7ba1\u63a7\u3001\u591a\u79df\u6237\u3001\u667a\u80fd\u8bca\u65ad\u7b49\u8ba1\u7b97\u6cbb\u7406\u7279\u6027\u3002</p>\n<h4>\u4f18\u70b9</h4>\n<p>\u4ee5\u5fae\u4f17\u94f6\u884c\u4e0e Linkis \u540c\u65f6\u5f00\u6e90\u7684\uff0c\u4ea4\u4e92\u5f0f\u6570\u636e\u5f00\u53d1\u63a2\u7d22\u5de5\u5177 Scriptis \u4e3a\u4f8b\uff0cScriptis \u7684\u5f00\u53d1\u4eba\u5458\u53ea\u9700\u5173\u6ce8 Web UI\u3001\u591a\u79cd\u6570\u636e\u5f00\u53d1\u8bed\u8a00\u652f\u6301\u3001\u811a\u672c\u7f16\u8f91\u529f\u80fd\u7b49\u7eaf\u524d\u7aef\u529f\u80fd\u5b9e\u73b0\uff0cLinkis \u5305\u529e\u4e86\u5176\u4ece\u5b58\u50a8\u8bfb\u5199\u3001\u8ba1\u7b97\u4efb\u52a1\u63d0\u4ea4\u6267\u884c\u3001\u4f5c\u4e1a\u72b6\u6001\u65e5\u5fd7\u66f4\u65b0\u3001\u8d44\u6e90\u7ba1\u63a7\u7b49\u7b49\u51e0\u4e4e\u6240\u6709\u540e\u53f0\u529f\u80fd\u3002\u57fa\u4e8e Linkis \u7684\u5927\u91cf\u8ba1\u7b97\u6cbb\u7406\u5c42\u80fd\u529b\u7684\u590d\u7528\uff0c\u5927\u5927\u964d\u4f4e\u4e86 Scriptis \u9879\u76ee\u7684\u5f00\u53d1\u6210\u672c\uff0c\u4f7f\u5f97 Scritpis \u76ee\u524d\u53ea\u9700\u8981\u6709\u9650\u7684\u524d\u7aef\u4eba\u5458\uff0c\u5373\u53ef\u5b8c\u6210\u7ef4\u62a4\u548c\u7248\u672c\u8fed\u4ee3\u5de5\u4f5c\u3002</p>\n<p>\u5982\u4e0b\u56fe 17\uff0cScriptis \u9879\u76ee 99.5% \u7684\u4ee3\u7801\uff0c\u90fd\u662f\u524d\u7aef\u7684 JS\u3001CSS \u4ee3\u7801\u3002\u540e\u53f0\u57fa\u672c\u5b8c\u5168\u590d\u7528 Linkis\u3002</p>\n<p><img alt=\"\" src=\"https://imgconvert.csdnimg.cn/aHR0cHM6Ly91cGxvYWQtaW1hZ2VzLmppYW5zaHUuaW8vdXBsb2FkX2ltYWdlcy81NDI2NzctZGRhMDU5ZGNjNDE1NGZiZA?x-oss-process=image/format,png\"/></p>\n&lt;center&gt;\u56fe 17&lt;/center&gt;\n<h3>\u5feb\u901f\u6269\u5c55\uff1a\u5bf9\u4e8e\u5e95\u5c42\u5f15\u64ce\uff0cLinkis \u5982\u4f55\u4ee5\u5f88\u5c0f\u7684\u5f00\u53d1\u91cf\uff0c\u5b9e\u73b0\u65b0\u5e95\u5c42\u5f15\u64ce\u5feb\u901f\u5bf9\u63a5</h3>\n<p><strong>\u6a21\u5757\u5316\u53ef\u63d2\u62d4\u7684\u8ba1\u7b97\u5f15\u64ce\u63a5\u5165\u8bbe\u8ba1\uff0c\u65b0\u5f15\u64ce\u63a5\u5165\u7b80\u5355\u5feb\u901f\u3002</strong></p>\n<p>\u5bf9\u4e8e\u5178\u578b\u4ea4\u4e92\u5f0f\u6a21\u5f0f\u8ba1\u7b97\u5f15\u64ce\uff08\u63d0\u4ea4\u4efb\u52a1\uff0c\u6267\u884c\uff0c\u8fd4\u56de\u7ed3\u679c\uff09\uff0c\u7528\u6237\u53ea\u9700\u8981 buildApplication \u548c executeLine \u8fd9 2 \u4e2a\u65b9\u6cd5\uff0c\u5c31\u53ef\u4ee5\u5b8c\u6210\u4e00\u4e2a\u65b0\u7684\u8ba1\u7b97\u5f15\u64ce\u63a5\u5165 Linkis\uff0c\u4ee3\u7801\u91cf\u6781\u5c11\u3002\u793a\u4f8b\u5982\u4e0b\u3002</p>\n<p>(1) AppManager \u90e8\u5206\uff1a\u7528\u6237\u5fc5\u987b\u5b9e\u73b0\u7684\u63a5\u53e3\u662f ApplicationBuilder\uff0c\u7528\u6765\u5c01\u88c5\u65b0\u5f15\u64ce\u8fde\u63a5\u5668\u5b9e\u4f8b\u542f\u52a8\u547d\u4ee4\u3002</p>\n<pre><code>1.  //\u7528\u6237\u5fc5\u987b\u5b9e\u73b0\u7684\u65b9\u6cd5: \u7528\u4e8e\u5c01\u88c5\u65b0\u5f15\u64ce\u8fde\u63a5\u5668\u5b9e\u4f8b\u542f\u52a8\u547d\u4ee4\n2.  def buildApplication(protocol:Protocol):ApplicationRequest\n</code></pre>\n<p>(2) EngineConn \u90e8\u5206\uff1a\u7528\u6237\u53ea\u9700\u5b9e\u73b0 executeLine \u65b9\u6cd5\uff0c\u5411\u65b0\u5f15\u64ce\u63d0\u4ea4\u6267\u884c\u8ba1\u7b97\u4efb\u52a1\uff1a</p>\n<pre><code>1.  //\u7528\u6237\u5fc5\u987b\u5b9e\u73b0\u7684\u65b9\u6cd5\uff1a\u7528\u4e8e\u8c03\u7528\u5e95\u5c42\u5f15\u64ce\u63d0\u4ea4\u6267\u884c\u8ba1\u7b97\u4efb\u52a1\n2.  def executeLine(context: EngineConnContext,code: String): ExecuteResponse\n</code></pre>\n<p>\u5f15\u64ce\u76f8\u5173\u5176\u4ed6\u529f\u80fd /\u65b9\u6cd5\u90fd\u5df2\u6709\u9ed8\u8ba4\u5b9e\u73b0\uff0c\u65e0\u5b9a\u5236\u5316\u9700\u6c42\u53ef\u76f4\u63a5\u590d\u7528\u3002</p>\n<h3>\u8fde\u901a\uff1aLinkis \u5982\u4f55\u6253\u901a\u5e94\u7528\u5b64\u5c9b</h3>\n<p>\u901a\u8fc7 Linkis \u63d0\u4f9b\u7684\u4e0a\u4e0b\u6587\u670d\u52a1\uff0c\u548c\u5b58\u50a8\u3001\u7269\u6599\u5e93\u670d\u52a1\uff0c\u63a5\u5165\u7684\u591a\u4e2a\u4e0a\u5c42\u5e94\u7528\u4e4b\u95f4\uff0c\u53ef\u8f7b\u677e\u5b9e\u73b0\u73af\u5883\u53d8\u91cf\u3001\u51fd\u6570\u3001\u7a0b\u5e8f\u5305\u3001\u6570\u636e\u6587\u4ef6\u7b49\uff0c\u76f8\u5173\u4fe1\u606f\u548c\u8d44\u6e90\u7684\u5171\u4eab\u548c\u590d\u7528\uff0c\u6253\u901a\u5e94\u7528\u5b64\u5c9b\u3002</p>\n<p><img alt=\"\" src=\"https://imgconvert.csdnimg.cn/aHR0cHM6Ly91cGxvYWQtaW1hZ2VzLmppYW5zaHUuaW8vdXBsb2FkX2ltYWdlcy81NDI2NzctNmQxMDdkN2Q0ZjgyMTY2Mg?x-oss-process=image/format,png\"/></p>\n&lt;center&gt;\u56fe 18&lt;/center&gt;\n<h4>Context Service \u4e0a\u4e0b\u6587\u670d\u52a1\u4ecb\u7ecd</h4>\n<p>Context Service \uff08 CS \uff09\u4e3a\u4e0d\u540c\u4e0a\u5c42\u5e94\u7528\u7cfb\u7edf\uff0c\u4e0d\u540c\u8ba1\u7b97\u4efb\u52a1\uff0c\u63d0\u4f9b\u4e86\u7edf\u4e00\u7684\u4e0a\u4e0b\u6587\u7ba1\u7406\u670d\u52a1\uff0c\u53ef\u5b9e\u73b0\u4e0a\u4e0b\u6587\u7684\u81ea\u5b9a\u4e49\u548c\u5171\u4eab\u3002\u5728 Linkis \u4e2d\uff0cCS \u9700\u8981\u7ba1\u7406\u7684\u4e0a\u4e0b\u6587\u5185\u5bb9\uff0c\u53ef\u5206\u4e3a\u5143\u6570\u636e\u4e0a\u4e0b\u6587\u3001\u6570\u636e\u4e0a\u4e0b\u6587\u548c\u8d44\u6e90\u4e0a\u4e0b\u6587 3 \u90e8\u5206\u3002</p>\n<p><img alt=\"\" src=\"https://imgconvert.csdnimg.cn/aHR0cHM6Ly91cGxvYWQtaW1hZ2VzLmppYW5zaHUuaW8vdXBsb2FkX2ltYWdlcy81NDI2NzctYzc4OGQwOTRhODFhZGZkYw?x-oss-process=image/format,png\"/></p>\n&lt;center&gt;\u56fe 19&lt;/center&gt;\n<p>\u5143\u6570\u636e\u4e0a\u4e0b\u6587\uff0c\u5b9a\u4e49\u4e86\u8ba1\u7b97\u4efb\u52a1\u4e2d\u5e95\u5c42\u5f15\u64ce\u5143\u6570\u636e\u7684\u8bbf\u95ee\u548c\u4f7f\u7528\u89c4\u8303\uff0c\u4e3b\u8981\u529f\u80fd\u5982\u4e0b\uff1a</p>\n<ul>\n<li>\n<p>\u63d0\u4f9b\u7528\u6237\u7684\u6240\u6709\u5143\u6570\u636e\u4fe1\u606f\u8bfb\u5199\u63a5\u53e3\uff08\u5305\u62ec Hive \u8868\u5143\u6570\u636e\u3001\u7ebf\u4e0a\u5e93\u8868\u5143\u6570\u636e\u3001\u5176\u4ed6 NoSQL \u5982 HBase\u3001Kafka \u7b49\u5143\u6570\u636e\uff09\u3002</p>\n</li>\n<li>\n<p>\u8ba1\u7b97\u4efb\u52a1\u5185\u6240\u9700\u5143\u6570\u636e\u7684\u6ce8\u518c\u3001\u7f13\u5b58\u548c\u7ba1\u7406\u3002</p>\n</li>\n<li>\n<p>\u6570\u636e\u4e0a\u4e0b\u6587\uff0c\u5b9a\u4e49\u4e86\u8ba1\u7b97\u4efb\u52a1\u4e2d\u6570\u636e\u6587\u4ef6\u7684\u8bbf\u95ee\u548c\u4f7f\u7528\u89c4\u8303\u3002\u7ba1\u7406\u6570\u636e\u6587\u4ef6\u7684\u5143\u6570\u636e\u3002</p>\n</li>\n<li>\n<p>\u8fd0\u884c\u65f6\u4e0a\u4e0b\u6587\uff0c\u7ba1\u7406\u5404\u79cd\u7528\u6237\u81ea\u5b9a\u4e49\u7684\u53d8\u91cf\u3001\u51fd\u6570\u3001\u4ee3\u7801\u6bb5\u3001\u7a0b\u5e8f\u5305\u7b49\u3002</p>\n</li>\n<li>\n<p>\u540c\u65f6 Linkis \u4e5f\u63d0\u4f9b\u4e86\u7edf\u4e00\u7684\u7269\u6599\u7ba1\u7406\u548c\u5b58\u50a8\u670d\u52a1\uff0c\u4e0a\u5c42\u5e94\u7528\u53ef\u6839\u636e\u9700\u8981\u5bf9\u63a5\uff0c\u4ece\u800c\u53ef\u5b9e\u73b0\u811a\u672c\u6587\u4ef6\u3001\u7a0b\u5e8f\u5305\u3001\u6570\u636e\u6587\u4ef6\u7b49\u5b58\u50a8\u5c42\u7684\u6253\u901a\u3002</p>\n</li>\n</ul>\n<h2>\u57fa\u4e8e\u8ba1\u7b97\u4e2d\u95f4\u4ef6 Linkis \u7684\u8ba1\u7b97\u6cbb\u7406 - \u7406\u4e4b\u8def\uff08 Insight \uff09</h2>\n<p>Linkis \u8ba1\u7b97\u6cbb\u7406\u7ec6\u5316\u7279\u6027\u8bbe\u8ba1\u4e0e\u5b9e\u73b0\u4ecb\u7ecd\uff0c\u5728\u9ad8\u5e76\u53d1\u3001\u9ad8\u53ef\u7528\u3001\u591a\u79df\u6237\u9694\u79bb\u3001\u8d44\u6e90\u7ba1\u63a7\u3001\u8ba1\u7b97\u4efb\u52a1\u7ba1\u7406\u7b56\u7565\u7b49\u65b9\u9762\uff0c\u505a\u4e86\u5927\u91cf\u7ec6\u5316\u8003\u91cf\u548c\u5b9e\u73b0\uff0c\u4fdd\u969c\u8ba1\u7b97\u4efb\u52a1\u5728\u590d\u6742\u6761\u4ef6\u4e0b\u6210\u529f\u6267\u884c\u3002</p>\n<h3>\u8ba1\u7b97\u4efb\u52a1\u7684\u9ad8\u5e76\u53d1\u652f\u6301</h3>\n<p>Linkis \u7684 Job \u57fa\u4e8e\u591a\u7ea7\u5f02\u6b65\u8bbe\u8ba1\u6a21\u5f0f\uff0c\u670d\u52a1\u95f4\u901a\u8fc7\u9ad8\u6548\u7684 RPC \u548c\u6d88\u606f\u961f\u5217\u6a21\u5f0f\u8fdb\u884c\u5feb\u901f\u901a\u4fe1\uff0c\u5e76\u53ef\u4ee5\u901a\u8fc7\u7ed9 Job \u6253\u4e0a\u521b\u5efa\u8005\u3001\u7528\u6237\u7b49\u591a\u79cd\u7c7b\u578b\u7684\u6807\u7b7e\u8fdb\u884c\u4efb\u52a1\u7684\u8f6c\u53d1\u548c\u9694\u79bb\u6765\u63d0\u9ad8 Job \u7684\u5e76\u53d1\u80fd\u529b\u3002\u901a\u8fc7 Linkis \u53ef\u4ee5\u505a\u5230 1 \u4e2a\u5165\u53e3\u670d\u52a1\uff08 Entrance \uff09\u540c\u65f6\u627f\u63a5\u8d85 1 \u4e07+ \u5728\u7ebf\u7684 Job \u8bf7\u6c42\u3002</p>\n<p>\u591a\u7ea7\u5f02\u6b65\u7684\u8bbe\u8ba1\u67b6\u6784\u56fe\u5982\u4e0b\uff1a</p>\n<p><img alt=\"\" src=\"https://imgconvert.csdnimg.cn/aHR0cHM6Ly91cGxvYWQtaW1hZ2VzLmppYW5zaHUuaW8vdXBsb2FkX2ltYWdlcy81NDI2NzctNDYxZTk2NWM2ZTljMDNmOQ?x-oss-process=image/format,png\"/></p>\n&lt;center&gt;\u56fe 20&lt;/center&gt;\n<p>\u5982\u4e0a\u56fe\u6240\u793a Job \u4ece GateWay \u5230 Entrance \u540e\uff0cJob \u4ece\u751f\u6210\u5230\u6267\u884c\uff0c\u5230\u4fe1\u606f\u63a8\u9001\u7ecf\u5386\u4e86\u591a\u4e2a\u7ebf\u7a0b\u6c60\uff0c\u6bcf\u4e2a\u73af\u8282\u90fd\u901a\u8fc7\u5f02\u6b65\u7684\u8bbe\u8ba1\u6a21\u5f0f\uff0c\u6bcf\u4e00\u4e2a\u7ebf\u7a0b\u6c60\u4e2d\u7684\u7ebf\u7a0b\u90fd\u91c7\u7528\u8fd0\u884c\u4e00\u6b21\u5373\u7ed3\u675f\u7684\u65b9\u5f0f\uff0c\u964d\u4f4e\u7ebf\u7a0b\u5f00\u9500\u3002\u6574\u4e2a Job \u4ece\u8bf7\u6c42\u2014\u6267\u884c\u2014\u5230\u4fe1\u606f\u63a8\u9001\u5168\u90fd\u5f02\u6b65\u5b8c\u6210\uff0c\u663e\u8457\u7684\u63d0\u9ad8\u4e86 Job \u7684\u5e76\u53d1\u80fd\u529b\u3002</p>\n<p>\u8fd9\u91cc\u9488\u5bf9\u8ba1\u7b97\u4efb\u52a1\u6700\u5173\u952e\u7684\u4e00\u73af Job \u8c03\u5ea6\u5c42\u8fdb\u884c\u8bf4\u660e\uff0c\u6d77\u91cf\u7528\u6237\u6210\u5343\u4e0a\u4e07\u7684\u5e76\u53d1\u4efb\u52a1\u7684\u538b\u529b\uff0c\u5728 Job \u8c03\u5ea6\u5c42\u4e2d\u662f\u5982\u4f55\u8fdb\u884c\u5b9e\u73b0\u7684\u5462\uff1f</p>\n<p>\u5728\u8bf7\u6c42\u63a5\u6536\u5c42\uff0c\u8bf7\u6c42\u63a5\u6536\u961f\u5217\u4e2d\uff0c\u4f1a\u7f13\u5b58\u524d\u7aef\u7528\u6237\u63d0\u4ea4\u8fc7\u6765\u7684\u6210\u5343\u4e0a\u4e07\u8ba1\u7b97\u4efb\u52a1\uff0c\u5e76\u6309\u7cfb\u7edf /\u7528\u6237\u5c42\u7ea7\u5212\u5206\u7684\u8c03\u5ea6\u7ec4\uff0c\u5206\u53d1\u5230\u4e0b\u6e38 Job \u8c03\u5ea6\u6c60\u4e2d\u7684\u5404\u4e2a\u8c03\u5ea6\u961f\u5217\uff1b\u5230 Job \u8c03\u5ea6\u5c42\uff0c\u591a\u4e2a\u8c03\u5ea6\u7ec4\u5bf9\u5e94\u7684\u8c03\u5ea6\u5668\uff0c\u4f1a\u540c\u65f6\u6d88\u8d39\u5bf9\u5e94\u7684\u8c03\u5ea6\u961f\u5217\uff0c\u83b7\u53d6 Job \u5e76\u63d0\u4ea4\u7ed9 Job \u6267\u884c\u6c60\u8fdb\u884c\u6267\u884c\u3002\u8fc7\u7a0b\u4e2d\u5927\u91cf\u4f7f\u7528\u4e86\u591a\u7ebf\u7a0b\u3001\u591a\u7ea7\u5f02\u6b65\u8c03\u5ea6\u6267\u884c\u7b49\u6280\u672f\u3002\u793a\u610f\u5982\u4e0b\u56fe 21\uff1a</p>\n<p><img alt=\"\" src=\"https://imgconvert.csdnimg.cn/aHR0cHM6Ly91cGxvYWQtaW1hZ2VzLmppYW5zaHUuaW8vdXBsb2FkX2ltYWdlcy81NDI2NzctNjMyN2JkNzlkN2Q3ZWE2Yw?x-oss-process=image/format,png\"/></p>\n&lt;center&gt;\u56fe 21&lt;/center&gt;\n<h3>\u5176\u4ed6\u7ec6\u5316\u7279\u6027</h3>\n<p>Linkis \u8fd8\u5728\u9ad8\u53ef\u7528\u3001\u591a\u79df\u6237\u9694\u79bb\u3001\u8d44\u6e90\u7ba1\u63a7\u3001\u8ba1\u7b97\u4efb\u52a1\u7ba1\u7406\u7b56\u7565\u7b49\u65b9\u9762\uff0c\u505a\u4e86\u5f88\u591a\u7ec6\u5316\u8003\u91cf\u548c\u5b9e\u73b0\u3002\u7bc7\u5e45\u6709\u9650\uff0c\u5728\u8fd9\u91cc\u4e0d\u518d\u8be6\u8ff0\u6bcf\u4e2a\u7ec6\u5316\u7279\u6027\u7684\u5b9e\u73b0\uff0c\u53ef\u53c2\u89c1 Github \u4e0a Linkis \u7684 Wiki\u3002\u540e\u7ee7\u6211\u4eec\u4f1a\u9488\u5bf9 Linkis \u7684\u8ba1\u7b97\u6cbb\u7406-\u7406\u4e4b\u8def\uff08 Insight \uff09\u7684\u7ec6\u5316\u7279\u6027\u76f8\u5173\u5185\u5bb9\uff0c\u518d\u505a\u4e13\u9898\u4ecb\u7ecd\u3002</p>\n<h2>\u7ed3\u8bed</h2>\n<p>\u57fa\u4e8e\u5982\u4e0a\u89e3\u8026\u3001\u590d\u7528\u3001\u5feb\u901f\u6269\u5c55\u3001\u8fde\u901a\u7b49\u67b6\u6784\u8bbe\u8ba1\u4f18\u70b9\uff0c\u53ca\u9ad8\u5e76\u53d1\u3001\u9ad8\u53ef\u7528\u3001\u591a\u79df\u6237\u9694\u79bb\u3001\u8d44\u6e90\u7ba1\u63a7\u7b49\u7ec6\u5316\u7279\u6027\u5b9e\u73b0\uff0c\u8ba1\u7b97\u4e2d\u95f4\u4ef6 Linkis \u5728\u5fae\u4f17\u751f\u4ea7\u73af\u5883\u7684\u5e94\u7528\u6548\u679c\u663e\u8457\u3002\u6781\u5927\u7684\u52a9\u529b\u4e86\u5fae\u4f17\u94f6\u884c\u4e00\u7ad9\u5f0f\u5927\u6570\u636e\u5e73\u53f0\u5957\u4ef6 WeDataSphere \u7684\u5feb\u901f\u6784\u5efa\uff0c\u4e14\u6784\u6210\u4e86 WeDataSphere \u5168\u8fde\u901a\u3001\u591a\u79df\u6237\u3001\u8d44\u6e90\u7ba1\u63a7\u7b49\u4f01\u4e1a\u7ea7\u7279\u6027\u7684\u57fa\u77f3\u3002</p>\n<p>Linkis \u5728\u5fae\u4f17\u5e94\u7528\u60c5\u51b5\u5982\u56fe 22\uff1a</p>\n<p><img alt=\"\" src=\"https://imgconvert.csdnimg.cn/aHR0cHM6Ly91cGxvYWQtaW1hZ2VzLmppYW5zaHUuaW8vdXBsb2FkX2ltYWdlcy81NDI2NzctM2E2NDJmN2NmNTM3MjZhOA?x-oss-process=image/format,png\"/></p>\n&lt;center&gt;\u56fe 22&lt;/center&gt;\n<p>\u6211\u4eec\u5df2\u5c06 Linkis \u5f00\u6e90\uff0cGithub repo \u5730\u5740\uff1a<a href=\"https://github.com/WeBankFinTech/Linkis\" rel=\"nofollow\">https://github.com/WeBankFinTech/Linkis</a>\u3002</p>\n<p>\u6b22\u8fce\u5bf9\u7c7b\u4f3c\u8ba1\u7b97\u6cbb\u7406\u95ee\u9898\u611f\u5174\u8da3\u7684\u540c\u5b66\uff0c\u53c2\u4e0e\u5230\u8ba1\u7b97\u4e2d\u95f4\u4ef6 Linkis \u7684\u793e\u533a\u534f\u4f5c\u4e2d\uff0c\u5171\u540c\u628a Linkis \u5efa\u8bbe\u5f97\u66f4\u52a0\u5b8c\u5584\u548c\u6613\u7528\u3002</p>\n<blockquote>\n<p>\u4f5c\u8005\u4ecb\u7ecd\uff1a\u90b8\u5e05\uff0c\u5fae\u4f17\u94f6\u884c\u5927\u6570\u636e\u5e73\u53f0\u8d1f\u8d23\u4eba\uff0c\u4e3b\u5bfc\u5fae\u4f17\u94f6\u884c WeDataSphere \u5927\u6570\u636e\u5e73\u53f0\u5957\u4ef6\u7684\u5efa\u8bbe\u8fd0\u8425\u4e0e\u5f00\u6e90\uff0c\u5177\u5907\u4e30\u5bcc\u7684\u5927\u6570\u636e\u5e73\u53f0\u5f00\u53d1\u5efa\u8bbe\u5b9e\u8df5\u7ecf\u9a8c\u3002</p>\n</blockquote>\n<p>\u672c\u6587\u662f\u300c\u5206\u5e03\u5f0f\u7cfb\u7edf\u524d\u6cbf\u6280\u672f\u300d\u4e13\u9898\u6587\u7ae0\uff0c\u76ee\u524d\u8be5\u4e13\u9898\u5728\u6301\u7eed\u66f4\u65b0\u4e2d\uff0c\u6b22\u8fce\u5927\u5bb6\u4fdd\u6301\u5173\u6ce8\ud83d\udc47</p>\n<p><img alt=\"\" src=\"https://img-blog.csdnimg.cn/20200102131935443.jpg?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L1RpREJfUGluZ0NBUA==,size_16,color_FFFFFF,t_70\"/></p>\n"
    }, 
    {
      "author": {
        "url": "https://www.v2ex.com/member/gancl", 
        "name": "gancl", 
        "avatar": "https://cdn.v2ex.com/avatar/757f/9aa4/53616_large.png?m=1401100020"
      }, 
      "url": "https://www.v2ex.com/t/577518", 
      "date_modified": "2019-06-26T01:30:45+00:00", 
      "content_html": "", 
      "date_published": "2019-06-26T01:30:19+00:00", 
      "title": "\u6296\u97f3\u4e0a\u5f88\u706b\u7684\u6570\u636e\u53ef\u89c6\u5316\u89c6\u9891\u91d1\u5341\u6570\u9e3d\u7684\u5236\u4f5c\u65b9\u6cd5,\u662f\u7528 unity3d \u505a\u7684\u8fd8\u662f moviepy \u8fd8\u662f\u7528\u52a8\u753b\u5236\u4f5c\u8f6f\u4ef6\u624b\u52a8\u505a\u7684?", 
      "id": "https://www.v2ex.com/t/577518"
    }, 
    {
      "author": {
        "url": "https://www.v2ex.com/member/ANEYE", 
        "name": "ANEYE", 
        "avatar": "https://cdn.v2ex.com/avatar/b1b9/4e88/386264_large.png?m=1568262469"
      }, 
      "url": "https://www.v2ex.com/t/557976", 
      "title": "\u8bfe\u7a0b\u9879\u76ee\u6709\u4ec0\u4e48\u4e91\u5e94\u7528\u5927\u6570\u636e\u9879\u76ee\u63a8\u8350\u5417", 
      "id": "https://www.v2ex.com/t/557976", 
      "date_published": "2019-04-23T08:49:31+00:00", 
      "content_html": "<p>\u5927\u4e09\u5728\u8bfb\uff0c\u672c\u5b66\u671f\u4e24\u95e8\u8bfe\u4e00\u95e8\u8bfe\u8981\u6c42\u505a\u5927\u6570\u636e\u9879\u76ee\u6bd4\u5982\u63a8\u8350\u7cfb\u7edf\u9884\u6d4b\u5206\u6790\uff0c\u53e6\u4e00\u95e8\u4e5f\u6709\u4e91\u5e94\u7528\u5927\u6570\u636e\u76f8\u5173\u9879\u76ee\uff0c\u6216\u8005\u5c0f\u7a0b\u5e8f\u4ec0\u4e48\u7684\u3002\u4e0d\u77e5\u9053\u505a\u5565\u80fd\u591f\u51fa\u5f69\u53c8\u4e0d\u592a\u96be\uff0c\u5404\u4f4d\u5e26\u54e5\u6709\u4ec0\u4e48\u63a8\u8350\u7684\u5417\uff1f</p>\n"
    }, 
    {
      "author": {
        "url": "https://www.v2ex.com/member/taurenshaman", 
        "name": "taurenshaman", 
        "avatar": "https://cdn.v2ex.com/avatar/6a0f/7ec1/40604_large.png?m=1371187588"
      }, 
      "url": "https://www.v2ex.com/t/287263", 
      "title": "\u8111\u7f51\u7edc\u7ec4\u56fe\u8c31\uff08 Brainnetome Atlas\uff09", 
      "id": "https://www.v2ex.com/t/287263", 
      "date_published": "2016-06-21T03:47:56+00:00", 
      "content_html": "\u6700\u65b0\u7248\uff1a <a target=\"_blank\" href=\"http://lore.chuci.info/taurenshaman/json/b607bbab7e0b4130a1cde86940f08466\" rel=\"nofollow\">http://lore.chuci.info/taurenshaman/json/b607bbab7e0b4130a1cde86940f08466</a>\r<br />\u89e3\u6790\u81ea\uff1a <a target=\"_blank\" href=\"http://atlas.brainnetome.org/\" rel=\"nofollow\">http://atlas.brainnetome.org/</a>\r<br />\r<br />\u6570\u636e\u5305\u62ec 6 \u4e2a\u5206\u7ec4\uff0c\uff1a\r<br />1. /js/bnatlasviewer/bnatlas_tree.json\r<br />2. /images/imglist.txt\r<br />3. \u4ece js \uff08/js/bnatlasviewer/bnatlasviewer.js \uff09\u4e2d\u89e3\u6790\u51fa\u7684 SVG \u56fe\u7247\r<br />4. /js/bnatlasviewer/data_centers.json\r<br />5. /js/bnatlasviewer/BDf_FDR05.json\r<br />6. /js/bnatlasviewer/PCf_FDR05.json\r<br />\r<br />\r<br />{\r<br />  &quot;title&quot;: &quot;\u8111\u7f51\u7edc\u7ec4\u56fe\u8c31\uff08 Brainnetome Atlas \uff09&quot;,\r<br />  &quot;url&quot;: &quot;http://atlas.brainnetome.org&quot;,\r<br />  &quot;groups&quot;: [\r<br />    {\r<br />      &quot;url&quot;: &quot;http://atlas.brainnetome.org/js/bnatlasviewer/data_centers.json&quot;,\r<br />      &quot;remark&quot;: &quot;converted to json format&quot;,\r<br />      &quot;data&quot;: [  /** .... **/      ]\r<br />    },\r<br />    /** .... **/\r<br />  ]\r<br />}"
    }, 
    {
      "author": {
        "url": "https://www.v2ex.com/member/Kai", 
        "name": "Kai", 
        "avatar": "https://cdn.v2ex.com/avatar/021b/bc7e/1024_large.png?m=1657391813"
      }, 
      "url": "https://www.v2ex.com/t/165115", 
      "title": "All of you working with Big Data, what is your Data?", 
      "id": "https://www.v2ex.com/t/165115", 
      "date_published": "2015-01-24T12:53:57+00:00", 
      "content_html": "<a target=\"_blank\" href=\"https://news.ycombinator.com/item?id=8939328\" rel=\"nofollow\">https://news.ycombinator.com/item?id=8939328</a>"
    }, 
    {
      "author": {
        "url": "https://www.v2ex.com/member/meteor2013", 
        "name": "meteor2013", 
        "avatar": "https://cdn.v2ex.com/avatar/e9b0/9450/30450_large.png?m=1355010412"
      }, 
      "url": "https://www.v2ex.com/t/98165", 
      "title": "\u8bf7\u95ee\u54ea\u91cc\u53ef\u4ee5\u67e5\u5230\u4e16\u754c\u5404\u56fd\u7684\u666e\u901a\u4eba\u4eec\u7684\u8dd1\u6b65\u6570\u636e\uff0c\u7761\u7720\u6570\u636e\uff0c\u8109\u640f\u6570\u636e\u7b49\u5065\u5eb7\u4fe1\u606f\uff1f\u6709\u8fd9\u6837\u7684\u7edf\u8ba1\u673a\u6784\u5b58\u5728\u5417\uff1f", 
      "id": "https://www.v2ex.com/t/98165", 
      "date_published": "2014-01-23T20:32:42+00:00", 
      "content_html": ""
    }, 
    {
      "author": {
        "url": "https://www.v2ex.com/member/meteor2013", 
        "name": "meteor2013", 
        "avatar": "https://cdn.v2ex.com/avatar/e9b0/9450/30450_large.png?m=1355010412"
      }, 
      "url": "https://www.v2ex.com/t/97330", 
      "date_modified": "2014-01-16T19:08:08+00:00", 
      "content_html": "\u5c0f\u516c\u53f8\uff0c\u4e2a\u4eba\u5c31\u6ca1\u6709\u529e\u6cd5\u73a9\u4e86\u5417\uff1f\u611f\u89c9\u81f3\u5c11\u6536\u96c6\u6570\u636e\u5c31\u6ca1\u529e\u6cd5\u3002<br />\u90a3\u4e48\uff0c\u5c0f\u516c\u53f8\uff0c\u4e2a\u4eba\u5728\u5927\u6570\u636e\u65f6\u4ee3\u80fd\u505a\u4e9b\u4ec0\u4e48\u5462\uff1f<br /><br />\u4f7f\u7528\u5927\u516c\u53f8\u63d0\u4f9b\u7684\u6570\u636e\u505a\u70b9\u5c0f\u7684\u5e94\u7528\uff1f", 
      "date_published": "2014-01-16T17:58:23+00:00", 
      "title": "\u7a81\u7136\u5728\u60f3\uff1a \u5927\u6570\u636e\u662f\u4e0d\u662f\u53ea\u6709\u50cf\u963f\u91cc\u5df4\u5df4\u8fd9\u6837\u7684\u5927\u4f01\u4e1a\u624d\u80fd\u73a9\uff1f\u5c0f\u516c\u53f8\uff0c\u4e2a\u4eba\u5728\u5927\u6570\u636e\u65f6\u4ee3\u80fd\u505a\u4e9b\u4ec0\u4e48\u5462\uff1f", 
      "id": "https://www.v2ex.com/t/97330"
    }, 
    {
      "author": {
        "url": "https://www.v2ex.com/member/ihacku", 
        "name": "ihacku", 
        "avatar": "https://cdn.v2ex.com/avatar/c8cd/63e1/3073_large.png?m=1675398764"
      }, 
      "url": "https://www.v2ex.com/t/88400", 
      "title": "Presto-Distributed SQL query engine for big data", 
      "id": "https://www.v2ex.com/t/88400", 
      "date_published": "2013-11-07T06:24:11+00:00", 
      "content_html": "<a href=\"http://prestodb.io/\" rel=\"nofollow\">http://prestodb.io/</a>"
    }, 
    {
      "author": {
        "url": "https://www.v2ex.com/member/Livid", 
        "name": "Livid", 
        "avatar": "https://cdn.v2ex.com/avatar/c4ca/4238/1_large.png?m=1776858751"
      }, 
      "url": "https://www.v2ex.com/t/49035", 
      "title": "Treasure Data \u83b7\u5f97 Matz \u6295\u8d44...", 
      "id": "https://www.v2ex.com/t/49035", 
      "date_published": "2012-09-30T02:25:04+00:00", 
      "content_html": "<a target=\"_blank\" href=\"http://techcrunch.com/2012/09/28/treasure-data-launches-cloud-based-data-warehouse-with-investment-from-ruby-creator-yukihiro-matz-matsumoto/\" rel=\"nofollow\">http://techcrunch.com/2012/09/28/treasure-data-launches-cloud-based-data-warehouse-with-investment-from-ruby-creator-yukihiro-matz-matsumoto/</a>"
    }, 
    {
      "author": {
        "url": "https://www.v2ex.com/member/Livid", 
        "name": "Livid", 
        "avatar": "https://cdn.v2ex.com/avatar/c4ca/4238/1_large.png?m=1776858751"
      }, 
      "url": "https://www.v2ex.com/t/48356", 
      "title": "\u6709\u4eba\u7528\u8fc7 Treasure Data \u4e48\uff1f", 
      "id": "https://www.v2ex.com/t/48356", 
      "date_published": "2012-09-20T17:23:22+00:00", 
      "content_html": "<a target=\"_blank\" href=\"http://www.treasure-data.com/\" rel=\"nofollow\">http://www.treasure-data.com/</a><br /><br />\u4e00\u4e2a\u770b\u8d77\u6765\u5f88\u4e0d\u9519\u7684 MapReduce \u4e91\u65b9\u6848\uff0c\u76ee\u524d\u5728 500GB \u4e4b\u5185\u90fd\u662f\u514d\u8d39\uff0c\u770b\u8d77\u6765\u4ed6\u4eec\u6253\u7b97\u5728 9.27 \u7684\u65f6\u5019\u516c\u5e03\u6b63\u5f0f\u7684\u5b9a\u4ef7\u3002"
    }, 
    {
      "author": {
        "url": "https://www.v2ex.com/member/Livid", 
        "name": "Livid", 
        "avatar": "https://cdn.v2ex.com/avatar/c4ca/4238/1_large.png?m=1776858751"
      }, 
      "url": "https://www.v2ex.com/t/48332", 
      "title": "Treasure Data", 
      "id": "https://www.v2ex.com/t/48332", 
      "date_published": "2012-09-20T13:02:21+00:00", 
      "content_html": "<a target=\"_blank\" href=\"http://www.treasure-data.com/\" rel=\"nofollow\">http://www.treasure-data.com/</a><br /><br />A promising Hadoop cloud."
    }
  ]
}