index.xml

<?xml version="1.0" encoding="utf-8" standalone="yes" ?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom">
  <channel>
    <title>Welcome to Xin&#39;s Homepage</title>
    <link>http://xincoder.github.io/</link>
      <atom:link href="http://xincoder.github.io/index.xml" rel="self" type="application/rss+xml" />
    <description>Welcome to Xin&#39;s Homepage</description>
    <generator>Wowchemy (https://wowchemy.com)</generator><language>en-us</language><lastBuildDate>Sat, 01 May 2021 00:00:00 +0000</lastBuildDate>
    <image>
      <url>http://xincoder.github.io/media/icon_hufdd866d90d76849587aac6fbf27da1ac_464_512x512_fill_lanczos_center_3.png</url>
      <title>Welcome to Xin&#39;s Homepage</title>
      <link>http://xincoder.github.io/</link>
    </image>
    
    <item>
      <title>SRNet: Spatial Relation Network for Efficient Single Stage Instance Segmentation in Videos</title>
      <link>http://xincoder.github.io/publication/2021acmm_srnet/</link>
      <pubDate>Sat, 01 May 2021 00:00:00 +0000</pubDate>
      <guid>http://xincoder.github.io/publication/2021acmm_srnet/</guid>
      <description>&lt;!-- &lt;div class=&#34;alert alert-note&#34;&gt;
  &lt;div&gt;
    Create your slides in Markdown - click the &lt;em&gt;Slides&lt;/em&gt; button to check out the example.
  &lt;/div&gt;
&lt;/div&gt;


Supplementary notes can be added here, including [code, math, and images](https://wowchemy.com/docs/writing-markdown-latex/). --&gt;
</description>
    </item>
    
    <item>
      <title>GPFS: A Graph-based Human Pose Forecasting System for Smart Home with Online Learning</title>
      <link>http://xincoder.github.io/publication/2021tosn_gpfs/</link>
      <pubDate>Mon, 01 Feb 2021 00:00:00 +0000</pubDate>
      <guid>http://xincoder.github.io/publication/2021tosn_gpfs/</guid>
      <description>&lt;!-- &lt;div class=&#34;alert alert-note&#34;&gt;
  &lt;div&gt;
    Create your slides in Markdown - click the &lt;em&gt;Slides&lt;/em&gt; button to check out the example.
  &lt;/div&gt;
&lt;/div&gt;


Supplementary notes can be added here, including [code, math, and images](https://wowchemy.com/docs/writing-markdown-latex/). --&gt;
</description>
    </item>
    
    <item>
      <title>Weakly-supervised Object Representation Learning for Few-shot Semantic Segmentation</title>
      <link>http://xincoder.github.io/publication/2021wacv_segment/</link>
      <pubDate>Sun, 10 Jan 2021 00:00:00 +0000</pubDate>
      <guid>http://xincoder.github.io/publication/2021wacv_segment/</guid>
      <description>&lt;!-- &lt;div class=&#34;alert alert-note&#34;&gt;
  &lt;div&gt;
    Create your slides in Markdown - click the &lt;em&gt;Slides&lt;/em&gt; button to check out the example.
  &lt;/div&gt;
&lt;/div&gt;


Supplementary notes can be added here, including [code, math, and images](https://wowchemy.com/docs/writing-markdown-latex/). --&gt;
</description>
    </item>
    
    <item>
      <title>GRIP&#43;&#43;: Enhanced Graph-based Interaction-aware Trajectory Prediction for Autonomous Driving</title>
      <link>http://xincoder.github.io/publication/2020arxiv_griplus/</link>
      <pubDate>Tue, 19 May 2020 00:00:00 +0000</pubDate>
      <guid>http://xincoder.github.io/publication/2020arxiv_griplus/</guid>
      <description>&lt;p&gt;GRIP++ ranked 1st on ApolloScape Trajectory Leaderboard at the time of publication.&lt;/p&gt;
&lt;!-- &lt;div class=&#34;alert alert-note&#34;&gt;
  &lt;div&gt;
    Create your slides in Markdown - click the &lt;em&gt;Slides&lt;/em&gt; button to check out the example.
  &lt;/div&gt;
&lt;/div&gt;


Supplementary notes can be added here, including [code, math, and images](https://wowchemy.com/docs/writing-markdown-latex/). --&gt;
</description>
    </item>
    
    <item>
      <title>Combining convolution and deconvolution for object detection</title>
      <link>http://xincoder.github.io/publication/patent_2020combining_objectdetection/</link>
      <pubDate>Mon, 20 Apr 2020 00:00:00 +0000</pubDate>
      <guid>http://xincoder.github.io/publication/patent_2020combining_objectdetection/</guid>
      <description>&lt;!-- &lt;div class=&#34;alert alert-note&#34;&gt;
  &lt;div&gt;
    Create your slides in Markdown - click the &lt;em&gt;Slides&lt;/em&gt; button to check out the example.
  &lt;/div&gt;
&lt;/div&gt;


Supplementary notes can be added here, including [code, math, and images](https://wowchemy.com/docs/writing-markdown-latex/). --&gt;
</description>
    </item>
    
    <item>
      <title>GRIP: Graph-based Interaction-aware Trajectory Prediction</title>
      <link>http://xincoder.github.io/publication/2019itsc_grip/</link>
      <pubDate>Sun, 28 Jul 2019 00:00:00 +0000</pubDate>
      <guid>http://xincoder.github.io/publication/2019itsc_grip/</guid>
      <description>&lt;!-- &lt;div class=&#34;alert alert-note&#34;&gt;
  &lt;div&gt;
    Create your slides in Markdown - click the &lt;em&gt;Slides&lt;/em&gt; button to check out the example.
  &lt;/div&gt;
&lt;/div&gt;


Supplementary notes can be added here, including [code, math, and images](https://wowchemy.com/docs/writing-markdown-latex/). --&gt;
</description>
    </item>
    
    <item>
      <title>DAC: Data-free Automatic Acceleration of Convolutional Networks</title>
      <link>http://xincoder.github.io/publication/2019wacv_dac/</link>
      <pubDate>Tue, 08 Jan 2019 00:00:00 +0000</pubDate>
      <guid>http://xincoder.github.io/publication/2019wacv_dac/</guid>
      <description>&lt;!-- &lt;div class=&#34;alert alert-note&#34;&gt;
  &lt;div&gt;
    Create your slides in Markdown - click the &lt;em&gt;Slides&lt;/em&gt; button to check out the example.
  &lt;/div&gt;
&lt;/div&gt;


Supplementary notes can be added here, including [code, math, and images](https://wowchemy.com/docs/writing-markdown-latex/). --&gt;
</description>
    </item>
    
    <item>
      <title>Data-free CNN acceleration [Research]</title>
      <link>http://xincoder.github.io/project/research_dac/</link>
      <pubDate>Mon, 05 Nov 2018 00:00:00 +0000</pubDate>
      <guid>http://xincoder.github.io/project/research_dac/</guid>
      <description>&lt;p&gt;Programming Language:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Python&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;This is a prototype of our &amp;ldquo;Data-free convolutional network acceleration&amp;rdquo; scheme. In this demo, we demonstrate the performance of our scheme in the task of multi-person pose estimation model and object detection. Please refer to our paper for more results and details.&lt;/p&gt;
&lt;p&gt;The following video shows a demo of our scheme.&lt;/p&gt;









  





&lt;video controls  &gt;
  &lt;source src=&#34;http://xincoder.github.io/project/research_dac/demo.mp4&#34; type=&#34;video/mp4&#34;&gt;
&lt;/video&gt;
&lt;hr&gt;
&lt;p&gt;Published paper:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Xin Li, Shuai Zhang, Bolan Jiang, Yingyong Qi, Mooi Choo Chuah, Ning Bi (2019). &lt;a href=&#34;../../publication/2019wacv_dac/&#34;&gt;DAC: Data-free Automatic Acceleration of Convolutional Networks&lt;/a&gt;. IEEE Winter Conference on Applications of Computer Vision (WACV). 2019.&lt;/li&gt;
&lt;/ul&gt;
</description>
    </item>
    
    <item>
      <title>LiveFace: A Multi-Task CNN for Fast Face-Authentication</title>
      <link>http://xincoder.github.io/publication/2018icmla_liveface/</link>
      <pubDate>Mon, 01 Oct 2018 00:00:00 +0000</pubDate>
      <guid>http://xincoder.github.io/publication/2018icmla_liveface/</guid>
      <description>&lt;!-- &lt;div class=&#34;alert alert-note&#34;&gt;
  &lt;div&gt;
    Create your slides in Markdown - click the &lt;em&gt;Slides&lt;/em&gt; button to check out the example.
  &lt;/div&gt;
&lt;/div&gt;


Supplementary notes can be added here, including [code, math, and images](https://wowchemy.com/docs/writing-markdown-latex/). --&gt;
</description>
    </item>
    
    <item>
      <title>Vehicle Detection[Self Driving]</title>
      <link>http://xincoder.github.io/project/selfdriving_vehicle_detection/</link>
      <pubDate>Sun, 02 Sep 2018 00:00:00 +0000</pubDate>
      <guid>http://xincoder.github.io/project/selfdriving_vehicle_detection/</guid>
      <description>&lt;h1 id=&#34;vehicle-detection-project&#34;&gt;&lt;strong&gt;Vehicle Detection Project&lt;/strong&gt;&lt;/h1&gt;
&lt;p&gt;Programming Language:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Python&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;The goals of this project are the following:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Perform a Histogram of Oriented Gradients (HOG) feature extraction on a labeled training set of images and train a classifier Linear SVM classifier&lt;/li&gt;
&lt;li&gt;Optionally, you can also apply a color transform and append binned color features, as well as histograms of color, to your HOG feature vector.&lt;/li&gt;
&lt;li&gt;Note: for those first two steps don&amp;rsquo;t forget to normalize your features and randomize a selection for training and testing.&lt;/li&gt;
&lt;li&gt;Implement a sliding-window technique and use your trained classifier to search for vehicles in images.&lt;/li&gt;
&lt;li&gt;Run your pipeline on a video stream (start with the test_video.mp4 and later implement on full project_video.mp4) and create a heat map of recurring detections frame by frame to reject outliers and follow detected vehicles.&lt;/li&gt;
&lt;li&gt;Estimate a bounding box for vehicles detected.&lt;/li&gt;
&lt;/ul&gt;
&lt;hr&gt;
&lt;h3 id=&#34;1-feature-extraction&#34;&gt;1. Feature Extraction&lt;/h3&gt;
&lt;p&gt;I extract the binned color, histogram of color and HOG as my feature and trained a SVC classifier using the concatenated feature.
The following shows the parameters that I used:&lt;/p&gt;
&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th style=&#34;text-align:center&#34;&gt;&lt;/th&gt;
&lt;th style=&#34;text-align:center&#34;&gt;&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;Feature&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;Setting&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;Color Space&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;YUV&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;orient&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;9&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;pix_per_cell&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;8&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;cell_per_block&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;2&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;hog_channel&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;&amp;lsquo;ALL&amp;rsquo;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;spatial_size&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;(16,16)&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;hist_bins&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;16&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;
&lt;p&gt;The SVC classifier is used to predict a give image is a are or not. The following image shows a positive and a negative sample.
















&lt;figure  &gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img alt=&#34;sample.png&#34; srcset=&#34;
               /project/selfdriving_vehicle_detection/sample_hu4975a1962bb17d2ff5bd5f1c08ca7a1c_59581_1a9bb69455e94e85d2f0899a757c9d6b.webp 400w,
               /project/selfdriving_vehicle_detection/sample_hu4975a1962bb17d2ff5bd5f1c08ca7a1c_59581_ae46e48a3332b59af1612a354f23d074.webp 760w,
               /project/selfdriving_vehicle_detection/sample_hu4975a1962bb17d2ff5bd5f1c08ca7a1c_59581_1200x1200_fit_q75_h2_lanczos_3.webp 1200w&#34;
               src=&#34;http://xincoder.github.io/project/selfdriving_vehicle_detection/sample_hu4975a1962bb17d2ff5bd5f1c08ca7a1c_59581_1a9bb69455e94e85d2f0899a757c9d6b.webp&#34;
               width=&#34;512&#34;
               height=&#34;288&#34;
               loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;/figure&gt;
&lt;/p&gt;
&lt;hr&gt;
&lt;p&gt;The computed histogram of color is shown as follow:
















&lt;figure  &gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img alt=&#34;color_hist.png&#34; srcset=&#34;
               /project/selfdriving_vehicle_detection/color_hist_hubb13abf26d88b75b75a81753379fbc2e_12171_c3239e6aa0b025a9a8f3ac070e5eeceb.webp 400w,
               /project/selfdriving_vehicle_detection/color_hist_hubb13abf26d88b75b75a81753379fbc2e_12171_16d9f07515e42c86163acc6374490860.webp 760w,
               /project/selfdriving_vehicle_detection/color_hist_hubb13abf26d88b75b75a81753379fbc2e_12171_1200x1200_fit_q75_h2_lanczos_3.webp 1200w&#34;
               src=&#34;http://xincoder.github.io/project/selfdriving_vehicle_detection/color_hist_hubb13abf26d88b75b75a81753379fbc2e_12171_c3239e6aa0b025a9a8f3ac070e5eeceb.webp&#34;
               width=&#34;512&#34;
               height=&#34;128&#34;
               loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;/figure&gt;
&lt;/p&gt;
&lt;hr&gt;
&lt;p&gt;The below figure shows the visualized HOG featgure.
















&lt;figure  &gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img alt=&#34;hog.png&#34; srcset=&#34;
               /project/selfdriving_vehicle_detection/hog_hu239401ce6ddad2cae211aeab52917c98_71997_4838bc0665627f5176a6f0023d276cb6.webp 400w,
               /project/selfdriving_vehicle_detection/hog_hu239401ce6ddad2cae211aeab52917c98_71997_df3f6c013aa4e74de58b6d6d9f7f35ae.webp 760w,
               /project/selfdriving_vehicle_detection/hog_hu239401ce6ddad2cae211aeab52917c98_71997_1200x1200_fit_q75_h2_lanczos_3.webp 1200w&#34;
               src=&#34;http://xincoder.github.io/project/selfdriving_vehicle_detection/hog_hu239401ce6ddad2cae211aeab52917c98_71997_4838bc0665627f5176a6f0023d276cb6.webp&#34;
               width=&#34;512&#34;
               height=&#34;288&#34;
               loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;/figure&gt;
&lt;/p&gt;
&lt;hr&gt;
&lt;p&gt;I applied sliding window on the bottom half part of the image.
















&lt;figure  &gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img alt=&#34;window.png&#34; srcset=&#34;
               /project/selfdriving_vehicle_detection/window_huc28e8406471c1c681b813622def959e9_119313_563497326ca28d17cab3f09a7e364683.webp 400w,
               /project/selfdriving_vehicle_detection/window_huc28e8406471c1c681b813622def959e9_119313_16ec6135f324690d89413392475be628.webp 760w,
               /project/selfdriving_vehicle_detection/window_huc28e8406471c1c681b813622def959e9_119313_1200x1200_fit_q75_h2_lanczos_3.webp 1200w&#34;
               src=&#34;http://xincoder.github.io/project/selfdriving_vehicle_detection/window_huc28e8406471c1c681b813622def959e9_119313_563497326ca28d17cab3f09a7e364683.webp&#34;
               width=&#34;460&#34;
               height=&#34;259&#34;
               loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;/figure&gt;
&lt;/p&gt;
&lt;hr&gt;
&lt;p&gt;For each window, the chosen features are extracted and passed to the pre-trained SVC classifier to predict if the current location is a car or not. Then, we generate a heat map based on the detected results and filter low value out.
The following shows the visualized detected results.&lt;/p&gt;
&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th style=&#34;text-align:center&#34;&gt;&lt;/th&gt;
&lt;th style=&#34;text-align:center&#34;&gt;&lt;/th&gt;
&lt;th style=&#34;text-align:center&#34;&gt;&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;Detected BBoxes&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;Heat Map&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;Final Detection&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;&lt;img width=&#39;640&#39; src=&#39;detect_1.jpg&#39;&gt;&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;&lt;img width=&#39;640&#39; src=&#39;1.png&#39;&gt;&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;&amp;lt;img width=&amp;lsquo;640&amp;rsquo;, src=&amp;lsquo;img_1.jpg&amp;rsquo;&amp;gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;&lt;img width=&#39;640&#39; src=&#39;detect_2.jpg&#39;&gt;&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;&lt;img width=&#39;640&#39; src=&#39;2.png&#39;&gt;&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;&amp;lt;img width=&amp;lsquo;640&amp;rsquo;, src=&amp;lsquo;img_2.jpg&amp;rsquo;&amp;gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;&lt;img width=&#39;640&#39; src=&#39;detect_3.jpg&#39;&gt;&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;&lt;img width=&#39;640&#39; src=&#39;3.png&#39;&gt;&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;&amp;lt;img width=&amp;lsquo;640&amp;rsquo;, src=&amp;lsquo;img_3.jpg&amp;rsquo;&amp;gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;&lt;img width=&#39;640&#39; src=&#39;detect_4.jpg&#39;&gt;&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;&lt;img width=&#39;640&#39; src=&#39;4.png&#39;&gt;&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;&amp;lt;img width=&amp;lsquo;640&amp;rsquo;, src=&amp;lsquo;img_4.jpg&amp;rsquo;&amp;gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;&lt;img width=&#39;640&#39; src=&#39;detect_5.jpg&#39;&gt;&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;&lt;img width=&#39;640&#39; src=&#39;5.png&#39;&gt;&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;&amp;lt;img width=&amp;lsquo;640&amp;rsquo;, src=&amp;lsquo;img_5.jpg&amp;rsquo;&amp;gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;&lt;img width=&#39;640&#39; src=&#39;detect_6.jpg&#39;&gt;&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;&lt;img width=&#39;640&#39; src=&#39;6.png&#39;&gt;&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;&amp;lt;img width=&amp;lsquo;640&amp;rsquo;, src=&amp;lsquo;img_6.jpg&amp;rsquo;&amp;gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;
&lt;h3 id=&#34;2-make-it-smooth&#34;&gt;2. Make it smooth.&lt;/h3&gt;
&lt;p&gt;Till now, the pipline works on single frame. However, there is no relationship between two continuous frames and the detected results are independent.
Thus, I came up with an idea that sonsidering several continuous frames to make the detected results more smoothly.
I record 3 latest heatmap and then sum them together before doing threshold. Using this way, the detected results become much more stable. Please refer the demo video.&lt;/p&gt;
&lt;h3 id=&#34;3-potential-issue&#34;&gt;3. Potential Issue.&lt;/h3&gt;
&lt;p&gt;The parameters of the current pipline are manually chosen, the trained model and chosen parameters may not work for all videos.&lt;/p&gt;
&lt;p&gt;Potential solution: using deep learning object detection models, e.g. SSD.&lt;/p&gt;
</description>
    </item>
    
    <item>
      <title>Detect Lane [Self Driving]</title>
      <link>http://xincoder.github.io/project/selfdriving_adv_findline/</link>
      <pubDate>Mon, 20 Aug 2018 00:00:00 +0000</pubDate>
      <guid>http://xincoder.github.io/project/selfdriving_adv_findline/</guid>
      <description>&lt;h1 id=&#34;advanced-lane-finding-project&#34;&gt;&lt;strong&gt;Advanced Lane Finding Project&lt;/strong&gt;&lt;/h1&gt;
&lt;p&gt;Programming Language:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Python&lt;/li&gt;
&lt;/ul&gt;









  





&lt;video controls  &gt;
  &lt;source src=&#34;http://xincoder.github.io/project/selfdriving_adv_findline/demo.mp4&#34; type=&#34;video/mp4&#34;&gt;
&lt;/video&gt;
&lt;p&gt;The goals of this project are the following:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Compute the camera calibration matrix and distortion coefficients given a set of chessboard images.&lt;/li&gt;
&lt;li&gt;Apply a distortion correction to raw images.&lt;/li&gt;
&lt;li&gt;Use color transforms, gradients, etc., to create a thresholded binary image.&lt;/li&gt;
&lt;li&gt;Apply a perspective transform to rectify binary image (&amp;ldquo;birds-eye view&amp;rdquo;).&lt;/li&gt;
&lt;li&gt;Detect lane pixels and fit to find the lane boundary.&lt;/li&gt;
&lt;li&gt;Determine the curvature of the lane and vehicle position with respect to center.&lt;/li&gt;
&lt;li&gt;Warp the detected lane boundaries back onto the original image.&lt;/li&gt;
&lt;li&gt;Output visual display of the lane boundaries and numerical estimation of lane curvature and vehicle position.&lt;/li&gt;
&lt;/ul&gt;
&lt;hr&gt;
&lt;h3 id=&#34;a-pipeline&#34;&gt;A. Pipeline.&lt;/h3&gt;
&lt;p&gt;My pipline consists of 7 steps as follows:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Step 1: Camera Calibration&lt;/li&gt;
&lt;li&gt;Step 2: Distortion correction&lt;/li&gt;
&lt;li&gt;Step 3: Detect lines based on color and gradient&lt;/li&gt;
&lt;li&gt;Step 4: Perspective transform&lt;/li&gt;
&lt;li&gt;Step 5: Detect lane lines&lt;/li&gt;
&lt;li&gt;Step 6: Determine the lane curvature&lt;/li&gt;
&lt;li&gt;Step 7: Determine vehicle offset from center&lt;/li&gt;
&lt;/ul&gt;
&lt;hr&gt;
&lt;h3 id=&#34;b-visualized-results&#34;&gt;B. Visualized Results&lt;/h3&gt;
&lt;p&gt;&lt;strong&gt;1. Camera calibration and Distortion correction:&lt;/strong&gt;
Image distortion occurs when a camera looks at 3D objects in the real world and transforms them into a 2D image; this transformation isn’t perfect. Distortion changes what the shape and size of these 3D objects appear to be.
The following is a sample of the processed results:&lt;/p&gt;
&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th style=&#34;text-align:center&#34;&gt;&lt;/th&gt;
&lt;th style=&#34;text-align:center&#34;&gt;&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;Input image&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;After Calibration&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;















&lt;figure  &gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img alt=&#34;0.Before_Calibration.jpg&#34; srcset=&#34;
               /project/selfdriving_adv_findline/Before_Calibration_hu5909208fb16d2109d0cdca2186f9358e_62300_9b41b4523f114eb4b92b93a4ac2581e2.webp 400w,
               /project/selfdriving_adv_findline/Before_Calibration_hu5909208fb16d2109d0cdca2186f9358e_62300_5715871c6ed2e5a7b05d2cdcbce79b4b.webp 760w,
               /project/selfdriving_adv_findline/Before_Calibration_hu5909208fb16d2109d0cdca2186f9358e_62300_1200x1200_fit_q75_h2_lanczos.webp 1200w&#34;
               src=&#34;http://xincoder.github.io/project/selfdriving_adv_findline/Before_Calibration_hu5909208fb16d2109d0cdca2186f9358e_62300_9b41b4523f114eb4b92b93a4ac2581e2.webp&#34;
               width=&#34;760&#34;
               height=&#34;428&#34;
               loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;/figure&gt;
&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;















&lt;figure  &gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img alt=&#34;1.After_Calibration.jpg&#34; srcset=&#34;
               /project/selfdriving_adv_findline/After_Calibration_hu4a7d233c13a18409b0b78ae5fc313105_130803_0329c9bae4400fb31543ab86510bdd9e.webp 400w,
               /project/selfdriving_adv_findline/After_Calibration_hu4a7d233c13a18409b0b78ae5fc313105_130803_8b48f2b94210a4ccf41eacc0ddda65cb.webp 760w,
               /project/selfdriving_adv_findline/After_Calibration_hu4a7d233c13a18409b0b78ae5fc313105_130803_1200x1200_fit_q75_h2_lanczos.webp 1200w&#34;
               src=&#34;http://xincoder.github.io/project/selfdriving_adv_findline/After_Calibration_hu4a7d233c13a18409b0b78ae5fc313105_130803_0329c9bae4400fb31543ab86510bdd9e.webp&#34;
               width=&#34;760&#34;
               height=&#34;428&#34;
               loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;/figure&gt;
&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;One can see that the lines on the left side of the image become straight after distortion.&lt;/td&gt;
&lt;td&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;
&lt;p&gt;&lt;strong&gt;2: Detect lines based on color and gradient:&lt;/strong&gt; Detect lines of the current lane based on color and gradient.&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;I first convert the image from RGB color space to HLS color space. The following shows the HLS color space.&lt;/li&gt;
&lt;/ul&gt;
&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th style=&#34;text-align:center&#34;&gt;&lt;/th&gt;
&lt;th style=&#34;text-align:center&#34;&gt;&lt;/th&gt;
&lt;th style=&#34;text-align:center&#34;&gt;&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;H channel&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;L channel&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;S channel&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;















&lt;figure  &gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img alt=&#34;2.HLS_h.jpg&#34; srcset=&#34;
               /project/selfdriving_adv_findline/HLS_h_hu58d88cff1baa750dd7a4e938ffd91c52_105428_6cb10ddb2ea20176adbc8c6b1c7b732e.webp 400w,
               /project/selfdriving_adv_findline/HLS_h_hu58d88cff1baa750dd7a4e938ffd91c52_105428_393f71e25f88ff187cb62d6bd5f386f8.webp 760w,
               /project/selfdriving_adv_findline/HLS_h_hu58d88cff1baa750dd7a4e938ffd91c52_105428_1200x1200_fit_q75_h2_lanczos.webp 1200w&#34;
               src=&#34;http://xincoder.github.io/project/selfdriving_adv_findline/HLS_h_hu58d88cff1baa750dd7a4e938ffd91c52_105428_6cb10ddb2ea20176adbc8c6b1c7b732e.webp&#34;
               width=&#34;760&#34;
               height=&#34;428&#34;
               loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;/figure&gt;
&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;















&lt;figure  &gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img alt=&#34;2.HLS_l.jpg&#34; srcset=&#34;
               /project/selfdriving_adv_findline/HLS_l_hu829c6fa94ec01f0d65bcece61c769ffb_170290_b20a252f599e75c129f4953e26978566.webp 400w,
               /project/selfdriving_adv_findline/HLS_l_hu829c6fa94ec01f0d65bcece61c769ffb_170290_ef68c1c6c449aa4a74d4a692330eaa6c.webp 760w,
               /project/selfdriving_adv_findline/HLS_l_hu829c6fa94ec01f0d65bcece61c769ffb_170290_1200x1200_fit_q75_h2_lanczos.webp 1200w&#34;
               src=&#34;http://xincoder.github.io/project/selfdriving_adv_findline/HLS_l_hu829c6fa94ec01f0d65bcece61c769ffb_170290_b20a252f599e75c129f4953e26978566.webp&#34;
               width=&#34;760&#34;
               height=&#34;428&#34;
               loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;/figure&gt;
&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;















&lt;figure  &gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img alt=&#34;2.HLS_s.jpg&#34; srcset=&#34;
               /project/selfdriving_adv_findline/HLS_s_hua1e8bc3b10460f621c6185659a6d1f70_160952_7247abdda4eab5ef70e07fb4da3a19ad.webp 400w,
               /project/selfdriving_adv_findline/HLS_s_hua1e8bc3b10460f621c6185659a6d1f70_160952_48b41de8ca57d5188af7ebfe7c2ac426.webp 760w,
               /project/selfdriving_adv_findline/HLS_s_hua1e8bc3b10460f621c6185659a6d1f70_160952_1200x1200_fit_q75_h2_lanczos.webp 1200w&#34;
               src=&#34;http://xincoder.github.io/project/selfdriving_adv_findline/HLS_s_hua1e8bc3b10460f621c6185659a6d1f70_160952_7247abdda4eab5ef70e07fb4da3a19ad.webp&#34;
               width=&#34;760&#34;
               height=&#34;428&#34;
               loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;/figure&gt;
&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;
&lt;ul&gt;
&lt;li&gt;Then, I do color selection on S channel and x gradient on L channel. The following shows the combined result:&lt;/li&gt;
&lt;/ul&gt;
&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th style=&#34;text-align:center&#34;&gt;&lt;/th&gt;
&lt;th style=&#34;text-align:center&#34;&gt;&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;Input Imgae&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;Color_Gradient Result&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;















&lt;figure  &gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img alt=&#34;0straight_lines1.jpg&#34; srcset=&#34;
               /project/selfdriving_adv_findline/straight_lines1_hu9475e40123017607cf3e014e405fbc8c_155253_360217771e6d6044345bb328d7a64d9b.webp 400w,
               /project/selfdriving_adv_findline/straight_lines1_hu9475e40123017607cf3e014e405fbc8c_155253_176bf5ade271534d4f08d9c27c1243ea.webp 760w,
               /project/selfdriving_adv_findline/straight_lines1_hu9475e40123017607cf3e014e405fbc8c_155253_1200x1200_fit_q75_h2_lanczos.webp 1200w&#34;
               src=&#34;http://xincoder.github.io/project/selfdriving_adv_findline/straight_lines1_hu9475e40123017607cf3e014e405fbc8c_155253_360217771e6d6044345bb328d7a64d9b.webp&#34;
               width=&#34;760&#34;
               height=&#34;428&#34;
               loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;/figure&gt;
&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;















&lt;figure  &gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img alt=&#34;3.color_gradient_combine.jpg&#34; srcset=&#34;
               /project/selfdriving_adv_findline/color_gradient_hue1747a3e0f6e2f88771a88afb007e724_207073_489c6cec23df5a0729b8c165d0f4b0ad.webp 400w,
               /project/selfdriving_adv_findline/color_gradient_hue1747a3e0f6e2f88771a88afb007e724_207073_e5e4e1c4f4883b077f58474e84964989.webp 760w,
               /project/selfdriving_adv_findline/color_gradient_hue1747a3e0f6e2f88771a88afb007e724_207073_1200x1200_fit_q75_h2_lanczos.webp 1200w&#34;
               src=&#34;http://xincoder.github.io/project/selfdriving_adv_findline/color_gradient_hue1747a3e0f6e2f88771a88afb007e724_207073_489c6cec23df5a0729b8c165d0f4b0ad.webp&#34;
               width=&#34;760&#34;
               height=&#34;428&#34;
               loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;/figure&gt;
&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;
&lt;ul&gt;
&lt;li&gt;After that, I do &lt;a href=&#34;https://docs.opencv.org/3.0-beta/doc/py_tutorials/py_imgproc/py_morphological_ops/py_morphological_ops.html&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;open operation&lt;/a&gt; on the previous result to remove noise. One can see that the tiny noise is remove.&lt;/li&gt;
&lt;/ul&gt;
&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th style=&#34;text-align:center&#34;&gt;&lt;/th&gt;
&lt;th style=&#34;text-align:center&#34;&gt;&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;Input Imgae&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;Open Operation&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;















&lt;figure  &gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img alt=&#34;3.color_gradient_combine.jpg&#34; srcset=&#34;
               /project/selfdriving_adv_findline/color_gradient_hue1747a3e0f6e2f88771a88afb007e724_207073_489c6cec23df5a0729b8c165d0f4b0ad.webp 400w,
               /project/selfdriving_adv_findline/color_gradient_hue1747a3e0f6e2f88771a88afb007e724_207073_e5e4e1c4f4883b077f58474e84964989.webp 760w,
               /project/selfdriving_adv_findline/color_gradient_hue1747a3e0f6e2f88771a88afb007e724_207073_1200x1200_fit_q75_h2_lanczos.webp 1200w&#34;
               src=&#34;http://xincoder.github.io/project/selfdriving_adv_findline/color_gradient_hue1747a3e0f6e2f88771a88afb007e724_207073_489c6cec23df5a0729b8c165d0f4b0ad.webp&#34;
               width=&#34;760&#34;
               height=&#34;428&#34;
               loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;/figure&gt;
&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;















&lt;figure  &gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img alt=&#34;3.open_operation.jpg&#34; srcset=&#34;
               /project/selfdriving_adv_findline/open_operation_hu08a859bdec93387bee731a63eaf2b693_166321_5f6495b0d6f611f33ad4b0ba74174b01.webp 400w,
               /project/selfdriving_adv_findline/open_operation_hu08a859bdec93387bee731a63eaf2b693_166321_4f1d2c6ced323e24e6c5e979fb2908f1.webp 760w,
               /project/selfdriving_adv_findline/open_operation_hu08a859bdec93387bee731a63eaf2b693_166321_1200x1200_fit_q75_h2_lanczos.webp 1200w&#34;
               src=&#34;http://xincoder.github.io/project/selfdriving_adv_findline/open_operation_hu08a859bdec93387bee731a63eaf2b693_166321_5f6495b0d6f611f33ad4b0ba74174b01.webp&#34;
               width=&#34;760&#34;
               height=&#34;428&#34;
               loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;/figure&gt;
&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;
&lt;p&gt;&lt;strong&gt;4. Perspective transform:&lt;/strong&gt; To detect lines, I convert the image to birdeye view. I chose the following source and destination points:&lt;/p&gt;
&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th style=&#34;text-align:center&#34;&gt;&lt;/th&gt;
&lt;th style=&#34;text-align:center&#34;&gt;&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;Source Points&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;Destination Points&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;200, 200&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;566, 470&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;980, 200&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;714, 470&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;980, 700&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;1055, 680&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;200, 700&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;253, 680&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;
&lt;p&gt;The following shows a sample. It is clear that two lines are parallel.&lt;/p&gt;
&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th style=&#34;text-align:center&#34;&gt;&lt;/th&gt;
&lt;th style=&#34;text-align:center&#34;&gt;&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;Input Imgae&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;Open Operation&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;















&lt;figure  &gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img alt=&#34;3.open_operation.jpg&#34; srcset=&#34;
               /project/selfdriving_adv_findline/open_operation_hu08a859bdec93387bee731a63eaf2b693_166321_5f6495b0d6f611f33ad4b0ba74174b01.webp 400w,
               /project/selfdriving_adv_findline/open_operation_hu08a859bdec93387bee731a63eaf2b693_166321_4f1d2c6ced323e24e6c5e979fb2908f1.webp 760w,
               /project/selfdriving_adv_findline/open_operation_hu08a859bdec93387bee731a63eaf2b693_166321_1200x1200_fit_q75_h2_lanczos.webp 1200w&#34;
               src=&#34;http://xincoder.github.io/project/selfdriving_adv_findline/open_operation_hu08a859bdec93387bee731a63eaf2b693_166321_5f6495b0d6f611f33ad4b0ba74174b01.webp&#34;
               width=&#34;760&#34;
               height=&#34;428&#34;
               loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;/figure&gt;
&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;















&lt;figure  &gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img alt=&#34;4.birds_eye_line.jpg&#34; srcset=&#34;
               /project/selfdriving_adv_findline/birds_eye_hu6033c61c9ea10d059a4e5392096d2869_75487_0fd17af6eb8f5d70b276e9d902b97d3b.webp 400w,
               /project/selfdriving_adv_findline/birds_eye_hu6033c61c9ea10d059a4e5392096d2869_75487_b452b5fcf0cfa64b2e0e37919309c096.webp 760w,
               /project/selfdriving_adv_findline/birds_eye_hu6033c61c9ea10d059a4e5392096d2869_75487_1200x1200_fit_q75_h2_lanczos.webp 1200w&#34;
               src=&#34;http://xincoder.github.io/project/selfdriving_adv_findline/birds_eye_hu6033c61c9ea10d059a4e5392096d2869_75487_0fd17af6eb8f5d70b276e9d902b97d3b.webp&#34;
               width=&#34;760&#34;
               height=&#34;428&#34;
               loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;/figure&gt;
&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;
&lt;p&gt;&lt;strong&gt;5. Detect lane lines:&lt;/strong&gt;&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;First, I calculate histogram on vertical direction.&lt;/li&gt;
&lt;/ul&gt;
&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th style=&#34;text-align:center&#34;&gt;&lt;/th&gt;
&lt;th style=&#34;text-align:center&#34;&gt;&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;Input Imgae&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;Histogram&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;















&lt;figure  &gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img alt=&#34;4.birds_eye_line.jpg&#34; srcset=&#34;
               /project/selfdriving_adv_findline/birds_eye_hu6033c61c9ea10d059a4e5392096d2869_75487_0fd17af6eb8f5d70b276e9d902b97d3b.webp 400w,
               /project/selfdriving_adv_findline/birds_eye_hu6033c61c9ea10d059a4e5392096d2869_75487_b452b5fcf0cfa64b2e0e37919309c096.webp 760w,
               /project/selfdriving_adv_findline/birds_eye_hu6033c61c9ea10d059a4e5392096d2869_75487_1200x1200_fit_q75_h2_lanczos.webp 1200w&#34;
               src=&#34;http://xincoder.github.io/project/selfdriving_adv_findline/birds_eye_hu6033c61c9ea10d059a4e5392096d2869_75487_0fd17af6eb8f5d70b276e9d902b97d3b.webp&#34;
               width=&#34;760&#34;
               height=&#34;428&#34;
               loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;/figure&gt;
&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;















&lt;figure  &gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img alt=&#34;5.histogram.jpg&#34; srcset=&#34;
               /project/selfdriving_adv_findline/hist_hu520cb2412bad4e2ee7178972c29a4720_27226_ca14a79ba1922fc663939744cba6a44d.webp 400w,
               /project/selfdriving_adv_findline/hist_hu520cb2412bad4e2ee7178972c29a4720_27226_c59e52eafa406fb0c804f2505b879730.webp 760w,
               /project/selfdriving_adv_findline/hist_hu520cb2412bad4e2ee7178972c29a4720_27226_1200x1200_fit_q75_h2_lanczos.webp 1200w&#34;
               src=&#34;http://xincoder.github.io/project/selfdriving_adv_findline/hist_hu520cb2412bad4e2ee7178972c29a4720_27226_ca14a79ba1922fc663939744cba6a44d.webp&#34;
               width=&#34;640&#34;
               height=&#34;480&#34;
               loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;/figure&gt;
&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;
&lt;ul&gt;
&lt;li&gt;Then, I detect lines and compute their polynomial functions.&lt;/li&gt;
&lt;/ul&gt;
&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th style=&#34;text-align:center&#34;&gt;&lt;/th&gt;
&lt;th style=&#34;text-align:center&#34;&gt;&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;Input Imgae&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;Detect Lines&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;















&lt;figure  &gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img alt=&#34;4.birds_eye_line.jpg&#34; srcset=&#34;
               /project/selfdriving_adv_findline/birds_eye_hu6033c61c9ea10d059a4e5392096d2869_75487_0fd17af6eb8f5d70b276e9d902b97d3b.webp 400w,
               /project/selfdriving_adv_findline/birds_eye_hu6033c61c9ea10d059a4e5392096d2869_75487_b452b5fcf0cfa64b2e0e37919309c096.webp 760w,
               /project/selfdriving_adv_findline/birds_eye_hu6033c61c9ea10d059a4e5392096d2869_75487_1200x1200_fit_q75_h2_lanczos.webp 1200w&#34;
               src=&#34;http://xincoder.github.io/project/selfdriving_adv_findline/birds_eye_hu6033c61c9ea10d059a4e5392096d2869_75487_0fd17af6eb8f5d70b276e9d902b97d3b.webp&#34;
               width=&#34;760&#34;
               height=&#34;428&#34;
               loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;/figure&gt;
&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;















&lt;figure  &gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img alt=&#34;6.edge.jpg&#34; srcset=&#34;
               /project/selfdriving_adv_findline/edge_hued7d517ea887094a0252907e1378fa8b_155856_58bc2b3f6aa73a3333ec5cbe033cc4a8.webp 400w,
               /project/selfdriving_adv_findline/edge_hued7d517ea887094a0252907e1378fa8b_155856_804278ea02b0c5a84614aef90b3a39b6.webp 760w,
               /project/selfdriving_adv_findline/edge_hued7d517ea887094a0252907e1378fa8b_155856_1200x1200_fit_q75_h2_lanczos.webp 1200w&#34;
               src=&#34;http://xincoder.github.io/project/selfdriving_adv_findline/edge_hued7d517ea887094a0252907e1378fa8b_155856_58bc2b3f6aa73a3333ec5cbe033cc4a8.webp&#34;
               width=&#34;760&#34;
               height=&#34;428&#34;
               loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;/figure&gt;
&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;
&lt;p&gt;&lt;strong&gt;6. Visualize final result:&lt;/strong&gt;&lt;/p&gt;
&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th style=&#34;text-align:center&#34;&gt;&lt;/th&gt;
&lt;th style=&#34;text-align:center&#34;&gt;&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;Input Imgae&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;Detect Lines&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;















&lt;figure  &gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img alt=&#34;0straight_lines1.jpg&#34; srcset=&#34;
               /project/selfdriving_adv_findline/straight_lines1_hu9475e40123017607cf3e014e405fbc8c_155253_360217771e6d6044345bb328d7a64d9b.webp 400w,
               /project/selfdriving_adv_findline/straight_lines1_hu9475e40123017607cf3e014e405fbc8c_155253_176bf5ade271534d4f08d9c27c1243ea.webp 760w,
               /project/selfdriving_adv_findline/straight_lines1_hu9475e40123017607cf3e014e405fbc8c_155253_1200x1200_fit_q75_h2_lanczos.webp 1200w&#34;
               src=&#34;http://xincoder.github.io/project/selfdriving_adv_findline/straight_lines1_hu9475e40123017607cf3e014e405fbc8c_155253_360217771e6d6044345bb328d7a64d9b.webp&#34;
               width=&#34;760&#34;
               height=&#34;428&#34;
               loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;/figure&gt;
&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;















&lt;figure  &gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img alt=&#34;result.jpg&#34; srcset=&#34;
               /project/selfdriving_adv_findline/result_huba0a951122241e474b34a3fc57363f12_216296_d585c68c0149697fbb4e12e67e92f475.webp 400w,
               /project/selfdriving_adv_findline/result_huba0a951122241e474b34a3fc57363f12_216296_767f647a0bb8dcffb046dbd1ad3271c1.webp 760w,
               /project/selfdriving_adv_findline/result_huba0a951122241e474b34a3fc57363f12_216296_1200x1200_fit_q75_h2_lanczos.webp 1200w&#34;
               src=&#34;http://xincoder.github.io/project/selfdriving_adv_findline/result_huba0a951122241e474b34a3fc57363f12_216296_d585c68c0149697fbb4e12e67e92f475.webp&#34;
               width=&#34;760&#34;
               height=&#34;428&#34;
               loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;/figure&gt;
&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;
&lt;hr&gt;
&lt;h3 id=&#34;potential-improvements&#34;&gt;Potential improvements&lt;/h3&gt;
&lt;ol&gt;
&lt;li&gt;This pipeline may be not work perfectly if the line is not very clear. Thus, merging the results generated using different color space will improve the performance.&lt;/li&gt;
&lt;li&gt;The perspective transform matrix may be not exactly same for different camera. Thus, we need a smarter way to calculate it automatically.&lt;/li&gt;
&lt;/ol&gt;
</description>
    </item>
    
    <item>
      <title>Drive in An Emulator [Self Driving]</title>
      <link>http://xincoder.github.io/project/selfdriving_drive_emulator/</link>
      <pubDate>Fri, 03 Aug 2018 00:00:00 +0000</pubDate>
      <guid>http://xincoder.github.io/project/selfdriving_drive_emulator/</guid>
      <description>&lt;h1 id=&#34;drive-in-a-simulator&#34;&gt;&lt;strong&gt;Drive in a simulator&lt;/strong&gt;&lt;/h1&gt;
&lt;p&gt;Programming Language:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Python&lt;/li&gt;
&lt;/ul&gt;









  





&lt;video controls  &gt;
  &lt;source src=&#34;http://xincoder.github.io/project/selfdriving_drive_emulator/demo.mp4&#34; type=&#34;video/mp4&#34;&gt;
&lt;/video&gt;
&lt;p&gt;The goals of this project are the following:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Use the simulator to collect data of good driving behavior&lt;/li&gt;
&lt;li&gt;Build, a convolution neural network in Keras that predicts steering angles from images&lt;/li&gt;
&lt;li&gt;Train and validate the model with a training and validation set&lt;/li&gt;
&lt;li&gt;Test that the model successfully drives around track one without leaving the road&lt;/li&gt;
&lt;li&gt;Summarize the results with a written report&lt;/li&gt;
&lt;/ul&gt;
&lt;hr&gt;
&lt;h2 id=&#34;section-1-model-architecture-and-training-strategy&#34;&gt;Section 1: Model Architecture and Training Strategy&lt;/h2&gt;
&lt;h3 id=&#34;1-model-architecture&#34;&gt;1. Model architecture&lt;/h3&gt;
&lt;p&gt;The following figure shows the architecture of my model.&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;First, I normalized all pixels of an input frame to [-1, 1].&lt;/li&gt;
&lt;li&gt;Second, I use Cropping2D to remove sky and the hood from every single frame.&lt;/li&gt;
&lt;li&gt;Third, 3 (5x5) convolutional layers are used to extract visual features from input frame. All convolutional layers have a RELU activation function to introduce nonlinearity. Each convolutional layer is followed by a MaxPooling2D layer to reduce the size of feature maps.&lt;/li&gt;
&lt;li&gt;Then, 1 flatten layer [model.py line 39] convert the feature maps to a vector and two fully connect layers are used to predict the final result.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;Predicting the angle of the steering wheel is a regression problem. Thus I used &amp;ldquo;mse&amp;rdquo; loss function.&lt;/p&gt;
&lt;p&gt;















&lt;figure  &gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img alt=&#34;sd.jpg&#34; srcset=&#34;
               /project/selfdriving_drive_emulator/sd_hub1e8d8ad8e16aee8aafc074e34b8fd48_195785_d76f3849838c210e2df2b0018c3a79c6.webp 400w,
               /project/selfdriving_drive_emulator/sd_hub1e8d8ad8e16aee8aafc074e34b8fd48_195785_597e20706fa172e5f569ffacf4cced31.webp 760w,
               /project/selfdriving_drive_emulator/sd_hub1e8d8ad8e16aee8aafc074e34b8fd48_195785_1200x1200_fit_q75_h2_lanczos.webp 1200w&#34;
               src=&#34;http://xincoder.github.io/project/selfdriving_drive_emulator/sd_hub1e8d8ad8e16aee8aafc074e34b8fd48_195785_d76f3849838c210e2df2b0018c3a79c6.webp&#34;
               width=&#34;303&#34;
               height=&#34;760&#34;
               loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;/figure&gt;
&lt;/p&gt;
&lt;h3 id=&#34;2-attempts-to-reduce-overfitting-in-the-model&#34;&gt;2. Attempts to reduce overfitting in the model&lt;/h3&gt;
&lt;ul&gt;
&lt;li&gt;&lt;strong&gt;Dropout:&lt;/strong&gt; The model contains dropout layers in order to reduce overfitting.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Data collection:&lt;/strong&gt; On both tracks, I drive the car Clockwise (2 loops) and counterclockwise (1 loop) and save data.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Training:&lt;/strong&gt; The model was trained and validated on different data sets to ensure that the model was not overfitting. The model was tested by running it through the simulator and ensuring that the vehicle could stay on the track.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Data Augmentation:&lt;/strong&gt; Every training frame has 50% probability to be horizontally flipped during training.&lt;/li&gt;
&lt;/ul&gt;
&lt;h3 id=&#34;3-model-parameter-tuning&#34;&gt;3. Model parameter tuning&lt;/h3&gt;
&lt;p&gt;The model used an adam optimizer, so the learning rate was not tuned manually.&lt;/p&gt;
&lt;h3 id=&#34;4-appropriate-training-data&#34;&gt;4. Appropriate training data&lt;/h3&gt;
&lt;p&gt;Training data was chosen to keep the vehicle driving on the road. I used a combination of center lane driving, recovering from the left and right sides of the road. Based on the ground truth:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;If the current angle is 0, then the correction value for data from the left and right camera is set to a small value.&lt;/li&gt;
&lt;li&gt;Otherwise, a larger correction value is used.
Using this way can avoid the car from performing as a snake.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;All data are randomly split into two subsets: 80% for training and 20% for validation. The model normalized input to [-1, 1] and crop out the top and the bottom parts to remove meaningless context.&lt;/p&gt;
&lt;p&gt;The following two figures are the original input and the result after cropped.&lt;/p&gt;
&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th style=&#34;text-align:center&#34;&gt;&lt;/th&gt;
&lt;th style=&#34;text-align:center&#34;&gt;&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;Original Input&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;Cropped Input&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;















&lt;figure  &gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img alt=&#34;original.jpg&#34; srcset=&#34;
               /project/selfdriving_drive_emulator/original_hu2d4c98e0176e40f450353bb1d77926f3_12786_661432fc67c43b0954f61c71aec63206.webp 400w,
               /project/selfdriving_drive_emulator/original_hu2d4c98e0176e40f450353bb1d77926f3_12786_d88f965afd2e2baa4ef0abfbb4e367f3.webp 760w,
               /project/selfdriving_drive_emulator/original_hu2d4c98e0176e40f450353bb1d77926f3_12786_1200x1200_fit_q75_h2_lanczos.webp 1200w&#34;
               src=&#34;http://xincoder.github.io/project/selfdriving_drive_emulator/original_hu2d4c98e0176e40f450353bb1d77926f3_12786_661432fc67c43b0954f61c71aec63206.webp&#34;
               width=&#34;320&#34;
               height=&#34;160&#34;
               loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;/figure&gt;
&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;















&lt;figure  &gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img alt=&#34;crop.jpg&#34; srcset=&#34;
               /project/selfdriving_drive_emulator/crop_hu9d3ac626c1d7dbb90526af418b26edbc_11027_48a7c659a7b2cc0007103ce8c26fe039.webp 400w,
               /project/selfdriving_drive_emulator/crop_hu9d3ac626c1d7dbb90526af418b26edbc_11027_af72391f79de7c88b7cb8de6d5f3d721.webp 760w,
               /project/selfdriving_drive_emulator/crop_hu9d3ac626c1d7dbb90526af418b26edbc_11027_1200x1200_fit_q75_h2_lanczos.webp 1200w&#34;
               src=&#34;http://xincoder.github.io/project/selfdriving_drive_emulator/crop_hu9d3ac626c1d7dbb90526af418b26edbc_11027_48a7c659a7b2cc0007103ce8c26fe039.webp&#34;
               width=&#34;320&#34;
               height=&#34;65&#34;
               loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;/figure&gt;
&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;
&lt;p&gt;The following shows the number of collected data. As you can see that, after using data from all 3 cameras and randomly horizontally flip frames, the distributation of the data is more balance.&lt;/p&gt;
&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th style=&#34;text-align:center&#34;&gt;&lt;/th&gt;
&lt;th style=&#34;text-align:center&#34;&gt;&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;Raw Data&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;Use all 3 cameras &amp;amp; Data Augmentation&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;















&lt;figure  &gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img alt=&#34;ori_dist.png&#34; srcset=&#34;
               /project/selfdriving_drive_emulator/ori_dist_hu4485c15b2113c9348d28a8e788915803_17092_a44665b874635e0c9cc60487c8cb7f05.webp 400w,
               /project/selfdriving_drive_emulator/ori_dist_hu4485c15b2113c9348d28a8e788915803_17092_7d54f7a282e1f0fa94a4b0fc530a81a7.webp 760w,
               /project/selfdriving_drive_emulator/ori_dist_hu4485c15b2113c9348d28a8e788915803_17092_1200x1200_fit_q75_h2_lanczos_3.webp 1200w&#34;
               src=&#34;http://xincoder.github.io/project/selfdriving_drive_emulator/ori_dist_hu4485c15b2113c9348d28a8e788915803_17092_a44665b874635e0c9cc60487c8cb7f05.webp&#34;
               width=&#34;640&#34;
               height=&#34;480&#34;
               loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;/figure&gt;
&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;















&lt;figure  &gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img alt=&#34;aug_dist.png&#34; srcset=&#34;
               /project/selfdriving_drive_emulator/aug_dist_hu0ac2f68ff88682a27a81998e0a2aa4d5_12254_77a26e076989badc2c8627320e16549c.webp 400w,
               /project/selfdriving_drive_emulator/aug_dist_hu0ac2f68ff88682a27a81998e0a2aa4d5_12254_961eb790b1f01d4f471cf77965b33009.webp 760w,
               /project/selfdriving_drive_emulator/aug_dist_hu0ac2f68ff88682a27a81998e0a2aa4d5_12254_1200x1200_fit_q75_h2_lanczos_3.webp 1200w&#34;
               src=&#34;http://xincoder.github.io/project/selfdriving_drive_emulator/aug_dist_hu0ac2f68ff88682a27a81998e0a2aa4d5_12254_77a26e076989badc2c8627320e16549c.webp&#34;
               width=&#34;640&#34;
               height=&#34;480&#34;
               loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;/figure&gt;
&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;
&lt;hr&gt;
&lt;h2 id=&#34;section-3-training-history-and-model-visualization&#34;&gt;Section 3: Training History and Model Visualization&lt;/h2&gt;
&lt;p&gt;This figure shows the loss history during training.
















&lt;figure  &gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img alt=&#34;loss.png&#34; srcset=&#34;
               /project/selfdriving_drive_emulator/loss_hu3078b3e4e80209cb97007719054332d0_36577_688a802913c33f1570e41cf3df90144b.webp 400w,
               /project/selfdriving_drive_emulator/loss_hu3078b3e4e80209cb97007719054332d0_36577_dfd34cf8bf0f49e79797b5116c916408.webp 760w,
               /project/selfdriving_drive_emulator/loss_hu3078b3e4e80209cb97007719054332d0_36577_1200x1200_fit_q75_h2_lanczos_3.webp 1200w&#34;
               src=&#34;http://xincoder.github.io/project/selfdriving_drive_emulator/loss_hu3078b3e4e80209cb97007719054332d0_36577_688a802913c33f1570e41cf3df90144b.webp&#34;
               width=&#34;640&#34;
               height=&#34;480&#34;
               loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;/figure&gt;
&lt;/p&gt;
&lt;p&gt;I also visulized the trained model and tried to understand what features the model learned.
















&lt;figure  &gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img alt=&#34;vis.jpg&#34; srcset=&#34;
               /project/selfdriving_drive_emulator/vis_hu2ad4cb61e4ffbc88ebfb6b5d15b86082_289580_148846ab6d261fc165320f2bb9e14322.webp 400w,
               /project/selfdriving_drive_emulator/vis_hu2ad4cb61e4ffbc88ebfb6b5d15b86082_289580_d7e2dd43b33fdf771fbc2cad324bb206.webp 760w,
               /project/selfdriving_drive_emulator/vis_hu2ad4cb61e4ffbc88ebfb6b5d15b86082_289580_1200x1200_fit_q75_h2_lanczos.webp 1200w&#34;
               src=&#34;http://xincoder.github.io/project/selfdriving_drive_emulator/vis_hu2ad4cb61e4ffbc88ebfb6b5d15b86082_289580_148846ab6d261fc165320f2bb9e14322.webp&#34;
               width=&#34;760&#34;
               height=&#34;190&#34;
               loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;/figure&gt;
&lt;/p&gt;
&lt;hr&gt;
&lt;h2 id=&#34;section-4-conclusion-and-future-work&#34;&gt;Section 4: Conclusion and Future work&lt;/h2&gt;
&lt;p&gt;In this project, I proposed a Convolutional Network to predict the angle of the steering wheel running in an emulator.&lt;/p&gt;
&lt;p&gt;The trained model successful negatives the car. Even in the second track (more challenging than the first one), the model still performs great.&lt;/p&gt;
&lt;p&gt;One of the possible drawback of the current model is taht it was only trained on the data collected from 2 tracks. Thus, it may cannot perform perfectly in other unseen scenario.&lt;/p&gt;
&lt;p&gt;Thus, in the future, I would like to train the model using more data collected from different scens.&lt;/p&gt;
</description>
    </item>
    
    <item>
      <title>Traffic Sign Recognition [Self Driving]</title>
      <link>http://xincoder.github.io/project/selfdriving_signclassification/</link>
      <pubDate>Mon, 16 Jul 2018 00:00:00 +0000</pubDate>
      <guid>http://xincoder.github.io/project/selfdriving_signclassification/</guid>
      <description>&lt;h1 id=&#34;traffic-sign-recognition&#34;&gt;&lt;strong&gt;Traffic Sign Recognition&lt;/strong&gt;&lt;/h1&gt;
&lt;p&gt;Programming Language:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Python&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;The goals of this project are the following:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Load the data set (see below for links to the project data set)&lt;/li&gt;
&lt;li&gt;Explore, summarize and visualize the data set&lt;/li&gt;
&lt;li&gt;Design, train and test a model architecture&lt;/li&gt;
&lt;li&gt;Use the model to make predictions on new images&lt;/li&gt;
&lt;li&gt;Analyze the softmax probabilities of the new images&lt;/li&gt;
&lt;li&gt;Summarize the results with a written report&lt;/li&gt;
&lt;/ul&gt;
&lt;hr&gt;
&lt;h2 id=&#34;step1-data-set-summary--exploration&#34;&gt;Step1: Data Set Summary &amp;amp; Exploration&lt;/h2&gt;
&lt;h4 id=&#34;1-a-basic-summary-of-the-data-set&#34;&gt;1. A basic summary of the data set.&lt;/h4&gt;
&lt;p&gt;I use Pickle data load function to read our dataset, and then use basic python function to analyze the dataset. In total, we have:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Training samples: 34,799&lt;/li&gt;
&lt;li&gt;Validation samples: 4,410&lt;/li&gt;
&lt;li&gt;Testing samples: 12,630&lt;/li&gt;
&lt;li&gt;Each sample (image) has (32, 32, 3) shape.&lt;/li&gt;
&lt;li&gt;We have 43 unique classes/labels:&lt;/li&gt;
&lt;/ul&gt;
&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th style=&#34;text-align:center&#34;&gt;&lt;/th&gt;
&lt;th style=&#34;text-align:center&#34;&gt;&lt;/th&gt;
&lt;th style=&#34;text-align:center&#34;&gt;&lt;/th&gt;
&lt;th style=&#34;text-align:center&#34;&gt;&lt;/th&gt;
&lt;th style=&#34;text-align:center&#34;&gt;&lt;/th&gt;
&lt;th style=&#34;text-align:center&#34;&gt;&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;ClassId&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;SignName&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;ClassId&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;SignName&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;ClassId&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;SignName&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;0&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;Speed limit (20km/h)&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;1&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;Speed limit (30km/h)&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;2&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;Speed limit (50km/h)&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;3&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;Speed limit (60km/h)&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;4&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;Speed limit (70km/h)&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;5&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;Speed limit (80km/h)&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;6&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;End of speed limit (80km/h)&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;7&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;Speed limit (100km/h)&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;8&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;Speed limit (120km/h)&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;9&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;No passing&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;10&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;No passing for vehicles over 3.5 metric tons&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;11&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;Right-of-way at the next intersection&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;12&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;Priority road&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;13&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;Yield&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;14&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;Stop&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;15&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;No vehicles&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;16&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;Vehicles over 3.5 metric tons prohibited&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;17&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;No entry&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;18&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;General caution&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;19&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;Dangerous curve to the left&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;20&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;Dangerous curve to the right&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;21&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;Double curve&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;22&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;Bumpy road&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;23&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;Slippery road&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;24&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;Road narrows on the right&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;25&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;Road work&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;26&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;Traffic signals&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;27&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;Pedestrians&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;28&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;Children crossing&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;29&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;Bicycles crossing&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;30&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;Beware of ice/snow&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;31&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;Wild animals crossing&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;32&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;End of all speed and passing limits&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;33&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;Turn right ahead&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;34&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;Turn left ahead&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;35&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;Ahead only&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;36&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;Go straight or right&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;37&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;Go straight or left&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;38&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;Keep right&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;39&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;Keep left&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;40&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;Roundabout mandatory&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;41&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;End of no passing&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;42&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;End of no passing by vehicles over 3.5 metric tons&lt;/td&gt;
&lt;td&gt;&lt;/td&gt;
&lt;td&gt;&lt;/td&gt;
&lt;td&gt;&lt;/td&gt;
&lt;td&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;
&lt;p&gt;















&lt;figure  &gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img alt=&#34;class_samples.png&#34; srcset=&#34;
               /project/selfdriving_signclassification/class_samples_huaeab329b455c9b7b8771b6f0046720b7_199239_27fbaf4a83f9a0f6db9b4b2cd7455b14.webp 400w,
               /project/selfdriving_signclassification/class_samples_huaeab329b455c9b7b8771b6f0046720b7_199239_808420021f2ac3d70d632893fd03fae4.webp 760w,
               /project/selfdriving_signclassification/class_samples_huaeab329b455c9b7b8771b6f0046720b7_199239_1200x1200_fit_q75_h2_lanczos_3.webp 1200w&#34;
               src=&#34;http://xincoder.github.io/project/selfdriving_signclassification/class_samples_huaeab329b455c9b7b8771b6f0046720b7_199239_27fbaf4a83f9a0f6db9b4b2cd7455b14.webp&#34;
               width=&#34;760&#34;
               height=&#34;569&#34;
               loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;/figure&gt;
&lt;/p&gt;
&lt;h4 id=&#34;2-here-is-an-exploratory-visualization-of-the-data-set&#34;&gt;2. Here is an exploratory visualization of the data set.&lt;/h4&gt;
&lt;p&gt;















&lt;figure  &gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img alt=&#34;data_bar.png&#34; srcset=&#34;
               /project/selfdriving_signclassification/data_bar_hu0f5ca4597522f72be28732ee917c34a8_14178_fd2da047a815bcf5312296379f196a8d.webp 400w,
               /project/selfdriving_signclassification/data_bar_hu0f5ca4597522f72be28732ee917c34a8_14178_c2a87ac5b95061ab139037a89aba4f5b.webp 760w,
               /project/selfdriving_signclassification/data_bar_hu0f5ca4597522f72be28732ee917c34a8_14178_1200x1200_fit_q75_h2_lanczos_3.webp 1200w&#34;
               src=&#34;http://xincoder.github.io/project/selfdriving_signclassification/data_bar_hu0f5ca4597522f72be28732ee917c34a8_14178_fd2da047a815bcf5312296379f196a8d.webp&#34;
               width=&#34;760&#34;
               height=&#34;210&#34;
               loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;/figure&gt;
&lt;/p&gt;
&lt;hr&gt;
&lt;h3 id=&#34;step2-design-and-test-a-model-architecture&#34;&gt;Step2: Design and Test a Model Architecture&lt;/h3&gt;
&lt;h4 id=&#34;1-image-data-preprocessing&#34;&gt;1. Image data preprocessing&lt;/h4&gt;
&lt;p&gt;I use two methods to do data augmentation (preprocessing).&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;Randomly rotation: all training images have 50% probability to be rotated within (-30, 30) angle. Rotation is needed considering that in daily life, the camera on a self-driving car may capture a traffice sign with a radom angle. Thus, by doing this help the model has the capability of handling such an application scenario.&lt;/li&gt;
&lt;li&gt;Randomly crop: all training images have 50% probability to be cropped from (0.8, 1) of width and height. After being cropped, the images will be resized back to 32x32. Cropping is necessary because the captured traffic signs have variant sizes. Thus, using cropping adds more training data. In addition, randomly cropping provides some smaples with width and height shifts.&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;Here are some processed images:
















&lt;figure  &gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img alt=&#34;augmentation.png&#34; srcset=&#34;
               /project/selfdriving_signclassification/augmentation_hud2883a39eb3f88d7d2614760cad77eba_77230_1efd78925926b67073d5ccff0a6dbc84.webp 400w,
               /project/selfdriving_signclassification/augmentation_hud2883a39eb3f88d7d2614760cad77eba_77230_21257824055c89dc97397fe1b6ef8fd2.webp 760w,
               /project/selfdriving_signclassification/augmentation_hud2883a39eb3f88d7d2614760cad77eba_77230_1200x1200_fit_q75_h2_lanczos_3.webp 1200w&#34;
               src=&#34;http://xincoder.github.io/project/selfdriving_signclassification/augmentation_hud2883a39eb3f88d7d2614760cad77eba_77230_1efd78925926b67073d5ccff0a6dbc84.webp&#34;
               width=&#34;760&#34;
               height=&#34;157&#34;
               loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;/figure&gt;
&lt;/p&gt;
&lt;h4 id=&#34;2-model-architecture&#34;&gt;2. Model architecture&lt;/h4&gt;
&lt;p&gt;My model consisted of the following layers:&lt;/p&gt;
&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th style=&#34;text-align:center&#34;&gt;Layer&lt;/th&gt;
&lt;th style=&#34;text-align:center&#34;&gt;Input_Size&lt;/th&gt;
&lt;th style=&#34;text-align:center&#34;&gt;Output_size&lt;/th&gt;
&lt;th style=&#34;text-align:center&#34;&gt;Description&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;Convolution(5x5)&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;32x32x3&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;28x28x64&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;(1x1) stride, VALID padding, Relu Activation&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;Convolution(3x3)&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;28x28x64&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;28x28x64&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;(1x1) stride, SAME padding, Relu Activation&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;Pooling&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;28x28x64&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;14x14x6&lt;/td&gt;
&lt;td&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;Convolution(5x5)&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;14x14x64&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;10x10x128&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;(1x1) stride, VALID padding, Relu Activation&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;Convolution(3x3)&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;10x10x128&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;10x10x128&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;(1x1) stride, SAME padding, Relu Activation&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;Pooling&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;10x10x128&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;5x5x12&lt;/td&gt;
&lt;td&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;Convolution(3x3)&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;5x5x128&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;3x3x256&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;(1x1) stride, VALID padding, Relu Activation&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;Pooling&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;3x3x256&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;1x1x25&lt;/td&gt;
&lt;td&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;Flatten&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;1x1x256&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;256&lt;/td&gt;
&lt;td&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;Fully Connected&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;256&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;128&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;Relu Activation&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;Dropout layer&lt;/td&gt;
&lt;td&gt;&lt;/td&gt;
&lt;td&gt;&lt;/td&gt;
&lt;td&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;Fully Connected&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;128&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;43&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;Softmax Activation&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;
&lt;h4 id=&#34;3-training-parameters&#34;&gt;3. Training parameters&lt;/h4&gt;
&lt;p&gt;I use the following experimental settings to train my model:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Optimizer: Adam&lt;/li&gt;
&lt;li&gt;Initialial Learning Rate: 0.001&lt;/li&gt;
&lt;li&gt;Batch Size: 128&lt;/li&gt;
&lt;li&gt;Training Epochs: 100&lt;/li&gt;
&lt;li&gt;Loss function: cross entropy.&lt;/li&gt;
&lt;/ul&gt;
&lt;h4 id=&#34;4-accuracy&#34;&gt;4. Accuracy&lt;/h4&gt;
&lt;p&gt;During training, I tracked the validation accuracy and only saved the weights achieved the highest validation accuracy. Within 100 training epochs, my architecture achieves the highest val_acc at 84 epoch. (All details about training process can be found in Cell [10])&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;span style=&#34;color:orange&#34;&gt;The accuracy on training set: &lt;strong&gt;99.9%&lt;/strong&gt;&lt;/span&gt;&lt;/li&gt;
&lt;li&gt;&lt;span style=&#34;color:orange&#34;&gt;The accuracy on validation set: &lt;strong&gt;96.2%&lt;/strong&gt;&lt;/span&gt;&lt;/li&gt;
&lt;li&gt;&lt;span style=&#34;color:orange&#34;&gt;The accuracy on testing set: &lt;strong&gt;93.4%&lt;/strong&gt;&lt;/span&gt;&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;The followings are the reasons why I chose such a model for this task:&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;In the first convolutional laeyrs, the model focuses on detecting fundamental features, e.g. lines, shapes, or textures. Thus, at the first two blocks (4 convolutional layers), I used two continuous convolutional layers to make sure that the model generates more useful basic features.&lt;/li&gt;
&lt;li&gt;Following that, a block with one convolutional layer is used to generate semantic information, e.g. arrows, circles, and etc.. Comparing to the basic features, we have more semantic features, thus, more filters were used in this layer.&lt;/li&gt;
&lt;li&gt;A fully connected layer is used to convert a feature map into a verctor for classification.&lt;/li&gt;
&lt;li&gt;One Dropout layer is used to avoid overfitting.&lt;/li&gt;
&lt;/ol&gt;
&lt;h3 id=&#34;test-a-model-on-new-images&#34;&gt;Test a Model on New Images&lt;/h3&gt;
&lt;h4 id=&#34;1-unseen-data-during-traing&#34;&gt;1. Unseen data during traing&lt;/h4&gt;
&lt;p&gt;Here are five German traffic signs that I found on the web:
















&lt;figure  &gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img alt=&#34;online.png&#34; srcset=&#34;
               /project/selfdriving_signclassification/online_hu6fbc8df78130d383a97b3112505cbaf4_42331_f1cb3965d8d23cfc9c158c93ff085305.webp 400w,
               /project/selfdriving_signclassification/online_hu6fbc8df78130d383a97b3112505cbaf4_42331_b8d3ed3169cac4aae0de205079208ef6.webp 760w,
               /project/selfdriving_signclassification/online_hu6fbc8df78130d383a97b3112505cbaf4_42331_1200x1200_fit_q75_h2_lanczos_3.webp 1200w&#34;
               src=&#34;http://xincoder.github.io/project/selfdriving_signclassification/online_hu6fbc8df78130d383a97b3112505cbaf4_42331_f1cb3965d8d23cfc9c158c93ff085305.webp&#34;
               width=&#34;378&#34;
               height=&#34;93&#34;
               loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;/figure&gt;
&lt;/p&gt;
&lt;h4 id=&#34;2-prediction-results-on-these-new-traffic-signs&#34;&gt;2. Prediction results on these new traffic signs.&lt;/h4&gt;
&lt;p&gt;Here are the results of the prediction:&lt;/p&gt;
&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th style=&#34;text-align:center&#34;&gt;&lt;/th&gt;
&lt;th style=&#34;text-align:center&#34;&gt;&lt;/th&gt;
&lt;th style=&#34;text-align:center&#34;&gt;&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;&lt;strong&gt;Image Ground Truth&lt;/strong&gt;&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;&lt;strong&gt;Prediction&lt;/strong&gt;&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;&lt;strong&gt;Predicted Correctly&lt;/strong&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;Go straight or right&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;Go straight or right&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;True&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;Speed limit (50km/h)&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;Speed limit (50km/h)&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;True&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;Stop&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;Yield&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;False&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;Road work&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;Road work&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;True&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;Turn right ahead&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;Turn right ahead&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;True&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;
&lt;p&gt;The model was able to correctly guess 4 of the 5 traffic signs, which gives an accuracy of 80%. We can see it drops a lot comparing to the accuracy on our testing dataset. The reason is three-fold:&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;The training dataset is not balance. For example, class 2 (Speed limit (50km/h)) has 2010 training samples, while class 14 (Stop) only consistes of 330 training samples.&lt;/li&gt;
&lt;li&gt;The downloaded images are much clearer than the images in our dataset, which may add more noise in the images.&lt;/li&gt;
&lt;li&gt;5 images are not enough to evaluate the performance of a trained model.&lt;/li&gt;
&lt;/ol&gt;
&lt;h4 id=&#34;3-prediction-details&#34;&gt;3. Prediction details&lt;/h4&gt;
&lt;p&gt;For the first image, the model is 100% sure that it is a &amp;ldquo;Go straight or right&amp;rdquo;. It is correctly predict this image.&lt;/p&gt;
&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th style=&#34;text-align:center&#34;&gt;&lt;/th&gt;
&lt;th style=&#34;text-align:center&#34;&gt;&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;&lt;strong&gt;Probability&lt;/strong&gt;&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;&lt;strong&gt;Prediction&lt;/strong&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;1.0000000&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;36 (Go straight or right)&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;2.8858561e-16&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;3 (Speed limit (60km/h))&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;2.1692284e-31&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;0 (Speed limit (20km/h))&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;7.6920753e-32&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;20 (Dangerous curve to the right)&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;2.2140123e-32&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;28 (Children crossing)&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;
&lt;p&gt;For the second image, the model is 100% sure that it is a &amp;ldquo;Speed limit (50km/h)&amp;rdquo;. It is correctly predict this image.&lt;/p&gt;
&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th style=&#34;text-align:center&#34;&gt;&lt;/th&gt;
&lt;th style=&#34;text-align:center&#34;&gt;&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;&lt;strong&gt;Probability&lt;/strong&gt;&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;&lt;strong&gt;Prediction&lt;/strong&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;9.9999976e-01&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;2 (Speed limit (50km/h))&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;2.8747601e-07&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;5 (Speed limit (80km/h))&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;2.2901984e-10&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;1 (Speed limit (30km/h))&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;2.2646303e-18&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;3 (Speed limit (60km/h))&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;4.5397839e-35&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;6 (End of speed limit (80km/h))&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;
&lt;p&gt;For the third image, the model is 100% sure that it is a &amp;ldquo;Yield&amp;rdquo;, while the ground truth is &amp;ldquo;Stop&amp;rdquo;. There are only 690 &amp;ldquo;Stop&amp;rdquo; training samples in our dataset, while &amp;ldquo;Yield&amp;rdquo; has 1290 training images. This is caused by our un-balance training data.&lt;/p&gt;
&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th style=&#34;text-align:center&#34;&gt;&lt;/th&gt;
&lt;th style=&#34;text-align:center&#34;&gt;&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;&lt;strong&gt;Probability&lt;/strong&gt;&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;&lt;strong&gt;Prediction&lt;/strong&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;1.0000000e+00&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;13 (Yield)&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;4.3748559e-12&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;28 (Children crossing)&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;1.5492816e-12&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;1 (Speed limit (30km/h))&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;1.0894177e-12&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;38 (Keep right)&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;6.2159755e-13&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;2 (Speed limit (50km/h))&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;
&lt;p&gt;For the fourth image, the model is 100% sure that it is a &amp;ldquo;Road work&amp;rdquo;. It is correctly predict this image.&lt;/p&gt;
&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th style=&#34;text-align:center&#34;&gt;&lt;/th&gt;
&lt;th style=&#34;text-align:center&#34;&gt;&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;&lt;strong&gt;Probability&lt;/strong&gt;&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;&lt;strong&gt;Prediction&lt;/strong&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;1.0000000e+00&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;25&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;0.0000000e+00&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;0 (Speed limit (20km/h))&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;0.0000000e+00&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;1 (Speed limit (30km/h))&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;0.0000000e+00&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;2 (Speed limit (50km/h))&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;0.0000000e+00&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;3 (Speed limit (60km/h))&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;
&lt;p&gt;For the fourth image, the model is 100% sure that it is a &amp;ldquo;Turn right ahead&amp;rdquo;. It is correctly predict this image.&lt;/p&gt;
&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th style=&#34;text-align:center&#34;&gt;&lt;/th&gt;
&lt;th style=&#34;text-align:center&#34;&gt;&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;&lt;strong&gt;Probability&lt;/strong&gt;&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;&lt;strong&gt;Prediction&lt;/strong&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;9.9085110e-01&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;33 (Turn right ahead)&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;8.0864038e-03&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;14 (Stop)&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;1.0624588e-03&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;4 (Speed limit (70km/h))&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;2.1159234e-11&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;13 (Yield)&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;7.8768702e-12&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;2 (Speed limit (50km/h))&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;
&lt;h3 id=&#34;visualizing-the-model&#34;&gt;Visualizing the model&lt;/h3&gt;
&lt;p&gt;I visulized the output of the first block. The circle and arrows are very clear.
















&lt;figure  &gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img alt=&#34;vis.png&#34; srcset=&#34;
               /project/selfdriving_signclassification/vis_hua3ea79a7ed0037084cb57388786915d9_42768_505e78c24db2173b826452c6f6531006.webp 400w,
               /project/selfdriving_signclassification/vis_hua3ea79a7ed0037084cb57388786915d9_42768_0baf066a6b144560a9f684552fdea062.webp 760w,
               /project/selfdriving_signclassification/vis_hua3ea79a7ed0037084cb57388786915d9_42768_1200x1200_fit_q75_h2_lanczos_3.webp 1200w&#34;
               src=&#34;http://xincoder.github.io/project/selfdriving_signclassification/vis_hua3ea79a7ed0037084cb57388786915d9_42768_505e78c24db2173b826452c6f6531006.webp&#34;
               width=&#34;760&#34;
               height=&#34;751&#34;
               loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;/figure&gt;
&lt;/p&gt;
</description>
    </item>
    
    <item>
      <title>Find Lane Lines [Self Driving]</title>
      <link>http://xincoder.github.io/project/selfdriving_findline/</link>
      <pubDate>Tue, 12 Jun 2018 00:00:00 +0000</pubDate>
      <guid>http://xincoder.github.io/project/selfdriving_findline/</guid>
      <description>&lt;p&gt;&lt;strong&gt;Finding Lane Lines on the Road&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;Programming Language:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Python&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;The goal of this project is to design a pipeline that finds lane lines on the road.









  





&lt;video controls  &gt;
  &lt;source src=&#34;http://xincoder.github.io/project/selfdriving_findline/result_demo.mp4&#34; type=&#34;video/mp4&#34;&gt;
&lt;/video&gt;&lt;/p&gt;
&lt;hr&gt;
&lt;h3 id=&#34;a-pipeline&#34;&gt;A. Pipeline.&lt;/h3&gt;
&lt;p&gt;My pipline consists of 7 steps as follows:&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;Color selection&lt;/li&gt;
&lt;li&gt;RGB image to gray image&lt;/li&gt;
&lt;li&gt;Gaussian Blur&lt;/li&gt;
&lt;li&gt;Edge detection (Canny)&lt;/li&gt;
&lt;li&gt;Select ROI&lt;/li&gt;
&lt;li&gt;Line detection (Hough)&lt;/li&gt;
&lt;li&gt;Extend detected lines&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;&lt;span style=&#34;color:orange&#34;&gt;This pipeline works on images and videos.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;1. Color selection:&lt;/strong&gt; Color selection is used to filter color so that we can remove those pixels which may become noise. E.g. patches on the road.
The following is a sample of the processed results:&lt;/p&gt;
&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th style=&#34;text-align:center&#34;&gt;&lt;/th&gt;
&lt;th style=&#34;text-align:center&#34;&gt;&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;Input image&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;Color Selection&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;















&lt;figure  &gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img alt=&#34;&#34; srcset=&#34;
               /project/selfdriving_findline/input_hua0d4d4b24f4da3a102e4452ac738b95f_482720_4d57b223f2bbbcebf4e6ffd111e21adc.webp 400w,
               /project/selfdriving_findline/input_hua0d4d4b24f4da3a102e4452ac738b95f_482720_3128d8e2c93ce5f6b99970a99933a43b.webp 760w,
               /project/selfdriving_findline/input_hua0d4d4b24f4da3a102e4452ac738b95f_482720_1200x1200_fit_q75_h2_lanczos.webp 1200w&#34;
               src=&#34;http://xincoder.github.io/project/selfdriving_findline/input_hua0d4d4b24f4da3a102e4452ac738b95f_482720_4d57b223f2bbbcebf4e6ffd111e21adc.webp&#34;
               width=&#34;760&#34;
               height=&#34;428&#34;
               loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;/figure&gt;
&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;















&lt;figure  &gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img alt=&#34;&#34; srcset=&#34;
               /project/selfdriving_findline/color_selection_hu31a2cb617c25f5bb400dc5a882cc8783_116050_f5c6d68c694ce22a805eab87af0d3a96.webp 400w,
               /project/selfdriving_findline/color_selection_hu31a2cb617c25f5bb400dc5a882cc8783_116050_f1ef1831f9887fc08f6965770fcaf6f9.webp 760w,
               /project/selfdriving_findline/color_selection_hu31a2cb617c25f5bb400dc5a882cc8783_116050_1200x1200_fit_q75_h2_lanczos.webp 1200w&#34;
               src=&#34;http://xincoder.github.io/project/selfdriving_findline/color_selection_hu31a2cb617c25f5bb400dc5a882cc8783_116050_f5c6d68c694ce22a805eab87af0d3a96.webp&#34;
               width=&#34;760&#34;
               height=&#34;428&#34;
               loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;/figure&gt;
&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;
&lt;p&gt;One can see that the lines of the current lane have already become very clear and there is no extra noise within the ROI (bottom middle part of the image).&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;2.RGB image to gray image:&lt;/strong&gt; This is a preprocessing for the flowing operations, e.g. edge detection.&lt;/p&gt;
&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th style=&#34;text-align:center&#34;&gt;&lt;/th&gt;
&lt;th style=&#34;text-align:center&#34;&gt;&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;Color Selection&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;Gray Image&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;















&lt;figure  &gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img alt=&#34;&#34; srcset=&#34;
               /project/selfdriving_findline/color_selection_hu31a2cb617c25f5bb400dc5a882cc8783_116050_f5c6d68c694ce22a805eab87af0d3a96.webp 400w,
               /project/selfdriving_findline/color_selection_hu31a2cb617c25f5bb400dc5a882cc8783_116050_f1ef1831f9887fc08f6965770fcaf6f9.webp 760w,
               /project/selfdriving_findline/color_selection_hu31a2cb617c25f5bb400dc5a882cc8783_116050_1200x1200_fit_q75_h2_lanczos.webp 1200w&#34;
               src=&#34;http://xincoder.github.io/project/selfdriving_findline/color_selection_hu31a2cb617c25f5bb400dc5a882cc8783_116050_f5c6d68c694ce22a805eab87af0d3a96.webp&#34;
               width=&#34;760&#34;
               height=&#34;428&#34;
               loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;/figure&gt;
&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;















&lt;figure  &gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img alt=&#34;&#34; srcset=&#34;
               /project/selfdriving_findline/gray_hu914336533fefe4f70747732bafdd6eb4_94606_da96734ce78f2726e158466e24e31e59.webp 400w,
               /project/selfdriving_findline/gray_hu914336533fefe4f70747732bafdd6eb4_94606_bdedbfbf818647a62fc4d179bf3aceb3.webp 760w,
               /project/selfdriving_findline/gray_hu914336533fefe4f70747732bafdd6eb4_94606_1200x1200_fit_q75_h2_lanczos.webp 1200w&#34;
               src=&#34;http://xincoder.github.io/project/selfdriving_findline/gray_hu914336533fefe4f70747732bafdd6eb4_94606_da96734ce78f2726e158466e24e31e59.webp&#34;
               width=&#34;760&#34;
               height=&#34;428&#34;
               loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;/figure&gt;
&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;
&lt;p&gt;&lt;strong&gt;3. Gaussian Blur:&lt;/strong&gt; This operation is also for edge detection. Using Gaussain Blur helps to remove noise from the current image.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;4. Edge detection (Canny):&lt;/strong&gt; We detect lines using Canny Transform.&lt;/p&gt;
&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th style=&#34;text-align:center&#34;&gt;&lt;/th&gt;
&lt;th style=&#34;text-align:center&#34;&gt;&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;Gaussian Blur&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;Edge Detection&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;















&lt;figure  &gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img alt=&#34;&#34; srcset=&#34;
               /project/selfdriving_findline/gray_hu914336533fefe4f70747732bafdd6eb4_94606_da96734ce78f2726e158466e24e31e59.webp 400w,
               /project/selfdriving_findline/gray_hu914336533fefe4f70747732bafdd6eb4_94606_bdedbfbf818647a62fc4d179bf3aceb3.webp 760w,
               /project/selfdriving_findline/gray_hu914336533fefe4f70747732bafdd6eb4_94606_1200x1200_fit_q75_h2_lanczos.webp 1200w&#34;
               src=&#34;http://xincoder.github.io/project/selfdriving_findline/gray_hu914336533fefe4f70747732bafdd6eb4_94606_da96734ce78f2726e158466e24e31e59.webp&#34;
               width=&#34;760&#34;
               height=&#34;428&#34;
               loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;/figure&gt;
&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;















&lt;figure  &gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img alt=&#34;&#34; srcset=&#34;
               /project/selfdriving_findline/edge_hu1633abf0146d20fa749c38f163246eef_100081_abd00d47d61f729162f5203289f3ef20.webp 400w,
               /project/selfdriving_findline/edge_hu1633abf0146d20fa749c38f163246eef_100081_0d96b43550ab56b71f364a61f0732876.webp 760w,
               /project/selfdriving_findline/edge_hu1633abf0146d20fa749c38f163246eef_100081_1200x1200_fit_q75_h2_lanczos.webp 1200w&#34;
               src=&#34;http://xincoder.github.io/project/selfdriving_findline/edge_hu1633abf0146d20fa749c38f163246eef_100081_abd00d47d61f729162f5203289f3ef20.webp&#34;
               width=&#34;760&#34;
               height=&#34;428&#34;
               loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;/figure&gt;
&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;
&lt;p&gt;&lt;strong&gt;5. Select ROI:&lt;/strong&gt; To remove the background, e.g. side of road or sky, we select a ROI that just in front of the driver (the bottom middle part of the current image).&lt;/p&gt;
&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th style=&#34;text-align:center&#34;&gt;&lt;/th&gt;
&lt;th style=&#34;text-align:center&#34;&gt;&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;Edge Detection&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;Select ROI&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;















&lt;figure  &gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img alt=&#34;&#34; srcset=&#34;
               /project/selfdriving_findline/edge_hu1633abf0146d20fa749c38f163246eef_100081_abd00d47d61f729162f5203289f3ef20.webp 400w,
               /project/selfdriving_findline/edge_hu1633abf0146d20fa749c38f163246eef_100081_0d96b43550ab56b71f364a61f0732876.webp 760w,
               /project/selfdriving_findline/edge_hu1633abf0146d20fa749c38f163246eef_100081_1200x1200_fit_q75_h2_lanczos.webp 1200w&#34;
               src=&#34;http://xincoder.github.io/project/selfdriving_findline/edge_hu1633abf0146d20fa749c38f163246eef_100081_abd00d47d61f729162f5203289f3ef20.webp&#34;
               width=&#34;760&#34;
               height=&#34;428&#34;
               loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;/figure&gt;
&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;















&lt;figure  &gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img alt=&#34;&#34; srcset=&#34;
               /project/selfdriving_findline/roi_hu08a859bdec93387bee731a63eaf2b693_62504_a0853cf6940b534ce1f9dea5b8ab18c4.webp 400w,
               /project/selfdriving_findline/roi_hu08a859bdec93387bee731a63eaf2b693_62504_302f054446553a4eda4c32106c5e28e3.webp 760w,
               /project/selfdriving_findline/roi_hu08a859bdec93387bee731a63eaf2b693_62504_1200x1200_fit_q75_h2_lanczos.webp 1200w&#34;
               src=&#34;http://xincoder.github.io/project/selfdriving_findline/roi_hu08a859bdec93387bee731a63eaf2b693_62504_a0853cf6940b534ce1f9dea5b8ab18c4.webp&#34;
               width=&#34;760&#34;
               height=&#34;428&#34;
               loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;/figure&gt;
&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;
&lt;p&gt;&lt;strong&gt;6. Line detection (Hough):&lt;/strong&gt; We use Hough Transform to detect lines. Instead of drawing the detected lines on the image, I implemented a function (detect_hough_line()) to return all detected lines, so that we can do further process.&lt;/p&gt;
&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th style=&#34;text-align:center&#34;&gt;&lt;/th&gt;
&lt;th style=&#34;text-align:center&#34;&gt;&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;Select ROI&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;Line Detection&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;















&lt;figure  &gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img alt=&#34;&#34; srcset=&#34;
               /project/selfdriving_findline/roi_hu08a859bdec93387bee731a63eaf2b693_62504_a0853cf6940b534ce1f9dea5b8ab18c4.webp 400w,
               /project/selfdriving_findline/roi_hu08a859bdec93387bee731a63eaf2b693_62504_302f054446553a4eda4c32106c5e28e3.webp 760w,
               /project/selfdriving_findline/roi_hu08a859bdec93387bee731a63eaf2b693_62504_1200x1200_fit_q75_h2_lanczos.webp 1200w&#34;
               src=&#34;http://xincoder.github.io/project/selfdriving_findline/roi_hu08a859bdec93387bee731a63eaf2b693_62504_a0853cf6940b534ce1f9dea5b8ab18c4.webp&#34;
               width=&#34;760&#34;
               height=&#34;428&#34;
               loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;/figure&gt;
&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;















&lt;figure  &gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img alt=&#34;&#34; srcset=&#34;
               /project/selfdriving_findline/detection_hu02570da8dff591db14e92b986bdd1dc3_85448_3300a8a4c164abe9efc79077defabde2.webp 400w,
               /project/selfdriving_findline/detection_hu02570da8dff591db14e92b986bdd1dc3_85448_71702a8876f50fc524911f4d052677c9.webp 760w,
               /project/selfdriving_findline/detection_hu02570da8dff591db14e92b986bdd1dc3_85448_1200x1200_fit_q75_h2_lanczos.webp 1200w&#34;
               src=&#34;http://xincoder.github.io/project/selfdriving_findline/detection_hu02570da8dff591db14e92b986bdd1dc3_85448_3300a8a4c164abe9efc79077defabde2.webp&#34;
               width=&#34;760&#34;
               height=&#34;428&#34;
               loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;/figure&gt;
&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;
&lt;p&gt;&lt;strong&gt;7. Extend detected lines:&lt;/strong&gt; The lines of a lane consists of solid lines and dashed lines. We want to use a long line to mark the boundaries (left line and right line) of the current lane.
Thus:&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;I implemented a function (namely, extend_lines()) to extend the lines, so that all lines can be extended to the bottom of the image and as far as possible.&lt;/li&gt;
&lt;li&gt;In this function, I also calculate the intersaction of these two lines, so that only the bottom part is marked with lines.&lt;/li&gt;
&lt;/ol&gt;
&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th style=&#34;text-align:center&#34;&gt;&lt;/th&gt;
&lt;th style=&#34;text-align:center&#34;&gt;&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;Line Detection&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;Extended lines (1)&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;















&lt;figure  &gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img alt=&#34;&#34; srcset=&#34;
               /project/selfdriving_findline/line_detection_hu02570da8dff591db14e92b986bdd1dc3_21361_7b8a97d7df9976f915d26b580f984928.webp 400w,
               /project/selfdriving_findline/line_detection_hu02570da8dff591db14e92b986bdd1dc3_21361_c11f9701bb62b6dd03eaedd45af7cda8.webp 760w,
               /project/selfdriving_findline/line_detection_hu02570da8dff591db14e92b986bdd1dc3_21361_1200x1200_fit_q75_h2_lanczos.webp 1200w&#34;
               src=&#34;http://xincoder.github.io/project/selfdriving_findline/line_detection_hu02570da8dff591db14e92b986bdd1dc3_21361_7b8a97d7df9976f915d26b580f984928.webp&#34;
               width=&#34;760&#34;
               height=&#34;428&#34;
               loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;/figure&gt;
&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;















&lt;figure  &gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img alt=&#34;&#34; srcset=&#34;
               /project/selfdriving_findline/extend_hu02570da8dff591db14e92b986bdd1dc3_117184_0f5b4d8eb70c7436d592d5b90fbc878e.webp 400w,
               /project/selfdriving_findline/extend_hu02570da8dff591db14e92b986bdd1dc3_117184_9ace4cf4e1b0c4f4a55da19a339f2a3e.webp 760w,
               /project/selfdriving_findline/extend_hu02570da8dff591db14e92b986bdd1dc3_117184_1200x1200_fit_q75_h2_lanczos.webp 1200w&#34;
               src=&#34;http://xincoder.github.io/project/selfdriving_findline/extend_hu02570da8dff591db14e92b986bdd1dc3_117184_0f5b4d8eb70c7436d592d5b90fbc878e.webp&#34;
               width=&#34;760&#34;
               height=&#34;428&#34;
               loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;/figure&gt;
&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;Calculate intersaction (2)&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;Merge with original image&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;















&lt;figure  &gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img alt=&#34;&#34; srcset=&#34;
               /project/selfdriving_findline/intersection_hu02570da8dff591db14e92b986bdd1dc3_98063_ce7b26c17c4697ae4d1576999e734aa7.webp 400w,
               /project/selfdriving_findline/intersection_hu02570da8dff591db14e92b986bdd1dc3_98063_c8698d748162131f809baac9b81a697a.webp 760w,
               /project/selfdriving_findline/intersection_hu02570da8dff591db14e92b986bdd1dc3_98063_1200x1200_fit_q75_h2_lanczos.webp 1200w&#34;
               src=&#34;http://xincoder.github.io/project/selfdriving_findline/intersection_hu02570da8dff591db14e92b986bdd1dc3_98063_ce7b26c17c4697ae4d1576999e734aa7.webp&#34;
               width=&#34;760&#34;
               height=&#34;428&#34;
               loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;/figure&gt;
&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;















&lt;figure  &gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img alt=&#34;1.jpg&#34; srcset=&#34;
               /project/selfdriving_findline/result_hud70679f95a7183b8632f028be4149f9c_613918_ced7f16a64ab10642ce2963542aa574c.webp 400w,
               /project/selfdriving_findline/result_hud70679f95a7183b8632f028be4149f9c_613918_5c6da9ee5b193cb055d75d38179ab860.webp 760w,
               /project/selfdriving_findline/result_hud70679f95a7183b8632f028be4149f9c_613918_1200x1200_fit_q75_h2_lanczos.webp 1200w&#34;
               src=&#34;http://xincoder.github.io/project/selfdriving_findline/result_hud70679f95a7183b8632f028be4149f9c_613918_ced7f16a64ab10642ce2963542aa574c.webp&#34;
               width=&#34;760&#34;
               height=&#34;428&#34;
               loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;/figure&gt;
&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;
&lt;hr&gt;
&lt;h3 id=&#34;b-potential-shortcomings-of-the-current-pipeline&#34;&gt;B. Potential shortcomings of the current pipeline&lt;/h3&gt;
&lt;ol&gt;
&lt;li&gt;One potential shortcoming would be caused by the fixed parameters in the current solution, e.g. Gaussian Blur parameters, Edge Detection parameters, Line Detection parameters, and so on. Such manually designed fixed parameters may not work in any application sceneria, e.g. ranning day or night.&lt;/li&gt;
&lt;li&gt;Another shortcoming would be caused by the first process (color selection). In real world, the color may change in different environment. Thus, in some specific environment, the current pipline may cannot detect any lines or detect too many lines (caused by noise).&lt;/li&gt;
&lt;/ol&gt;
&lt;hr&gt;
&lt;h3 id=&#34;c-suggest-possible-improvements-to-this-pipeline&#34;&gt;C. Suggest possible improvements to this pipeline&lt;/h3&gt;
&lt;ol&gt;
&lt;li&gt;For the first optential shortcoming, we would like to come up with an idea to adjust the parameters depening on the environment. Thus, the solution can be robust to the change of the environment.&lt;/li&gt;
&lt;li&gt;For the second optential shortcoming, we can use the same strategy as the first suggest. By automatically adjusting the paramters of color selection, the method should work. In addition, we can also replace the color selection with other preprocessing methods to avoid the sensitivity of chosen parameters.&lt;/li&gt;
&lt;/ol&gt;
&lt;hr&gt;
</description>
    </item>
    
    <item>
      <title></title>
      <link>http://xincoder.github.io/blogs/b_deep_learning/environment/</link>
      <pubDate>Fri, 18 May 2018 00:00:00 +0000</pubDate>
      <guid>http://xincoder.github.io/blogs/b_deep_learning/environment/</guid>
      <description>&lt;p&gt;In this blog, we are going to use &lt;a href=&#34;https://keras.io/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Keras&lt;/a&gt;(A Python Deep Learning Library) to implement our deep learning model. It is compatible with Python 2.7-3.5. Keras uses &lt;a href=&#34;https://www.tensorflow.org/install/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Tensorflow&lt;/a&gt;, &lt;a href=&#34;http://deeplearning.net/software/theano/install.html#install&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Theano&lt;/a&gt;, or &lt;a href=&#34;https://docs.microsoft.com/en-us/cognitive-toolkit/setup-cntk-on-your-machine&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;CNTK&lt;/a&gt; as its backend engines, so only need to install one of them.&lt;/p&gt;
&lt;h3 id=&#34;dependencies&#34;&gt;Dependencies:&lt;/h3&gt;
&lt;ol&gt;
&lt;li&gt;Keras 2.0.6 (or higher version).&lt;/li&gt;
&lt;li&gt;Python 2.7-3.5&lt;/li&gt;
&lt;li&gt;Tensorflow or Theano or CNTK&lt;/li&gt;
&lt;li&gt;HDF5&lt;/li&gt;
&lt;li&gt;h5py&lt;/li&gt;
&lt;li&gt;graphviz&lt;/li&gt;
&lt;li&gt;pydot&lt;/li&gt;
&lt;li&gt;cuDNN (only for running on GPU)&lt;/li&gt;
&lt;li&gt;opencv&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;&lt;strong&gt;HDF5&lt;/strong&gt; and &lt;strong&gt;h5py&lt;/strong&gt; libraries are used to save our model to disk. &lt;strong&gt;graphviz&lt;/strong&gt; and &lt;strong&gt;pydot&lt;/strong&gt; libraries are needed only when you want to plot model graphs to files (.png or .pdf).&lt;/p&gt;
&lt;p&gt;In addition, if your computer has one or multiple NVIDIA graph cards, you may want to install &lt;strong&gt;cuDNN&lt;/strong&gt; library to run Keras on GPU.&lt;/p&gt;
&lt;hr&gt;
&lt;p&gt;Next, I will show you how to setup the environment in &lt;a href=&#34;https://anaconda.org/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Anaconda&lt;/a&gt; on MacOS. (For other Operating Systems, please modify the corresponding links or commands.)&lt;/p&gt;
&lt;pre&gt;&lt;code class=&#34;language-Bash&#34;&gt;# Open a terminal and type the following commands

# Step 1: Download Anaconda
&amp;gt; curl -O https://repo.continuum.io/archive/Anaconda3-4.4.0-MacOSX-x86_64.sh

# Step 2: Install Anaconda (Use all default settings)
&amp;gt; bash Anaconda3-4.4.0-MacOSX-x86_64.sh

# Step 3: Restart your terminal

# Step 4: Create a virtual environment. (so that it will not mess up the existing settings) 
&amp;gt; conda create -n keras python=3.5

# Step 5: Install Tensorflow CPU version on Mac
&amp;gt; source activate keras
&amp;gt; pip install --upgrade https://storage.googleapis.com/tensorflow/mac/cpu/tensorflow-1.3.0-py3-none-any.whl
# If there is an error, please try it again.

# Step 6: Install Keras
&amp;gt; pip install keras

# Step 7: Install other Dependencies
&amp;gt; conda install HDF5
&amp;gt; conda install h5py
&amp;gt; pip install pydot
&amp;gt; pip install graphviz
&amp;gt; pip install pillow
&amp;gt; conda install -c https://conda.anaconda.org/menpo opencv3

# Step 8: Test
&amp;gt; python
from keras.models import Sequential
from keras.layers import Dense
from keras.utils import plot_model
model = Sequential()
model.add(Dense(10, input_shape=(700, 1)))
model.summary()
plot_model(model, to_file=&#39;abc.pdf&#39;, show_shapes=True)
exit()

# If you get some error like: install graphviz, you can try this command:
&amp;gt; brew install graphviz
&lt;/code&gt;&lt;/pre&gt;
</description>
    </item>
    
    <item>
      <title></title>
      <link>http://xincoder.github.io/blogs/b_deep_learning/neural_network/</link>
      <pubDate>Fri, 18 May 2018 00:00:00 +0000</pubDate>
      <guid>http://xincoder.github.io/blogs/b_deep_learning/neural_network/</guid>
      <description>&lt;p&gt;In this blog, we are going to use Neural Network to do image classification. The following figure shows a simple example of Neural Network. If you are interested in this field, please see this &lt;a href=&#34;https://neurophysics.ucsd.edu/courses/physics_171/annurev.neuro.28.061604.135703.pdf&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;review&lt;/a&gt; or recently this &lt;a href=&#34;http://www.sciencedirect.com/science/article/pii/S0959438814000130&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;review&lt;/a&gt;.
















&lt;figure  &gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img src=&#34;nn.png&#34; alt=&#34;&#34; loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;/figure&gt;
&lt;/p&gt;
&lt;hr&gt;
&lt;h3 id=&#34;1-a-simple-neural-network-on-mnist-datasethttpyannlecuncomexdbmnist-as-an-example&#34;&gt;1. A simple Neural Network on &lt;a href=&#34;http://yann.lecun.com/exdb/mnist/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;MNIST dataset&lt;/a&gt; as an example.&lt;/h3&gt;
&lt;p&gt;The MNIST database is a large database of handwritten digits. It contains 60,000 training images and 10,000 testing images. The Keras provides a convenience method for loading the MNIST dataset.&lt;/p&gt;
&lt;pre&gt;&lt;code class=&#34;language-Python&#34;&gt;# In this demo we design a Neural Network with 1 hidden layers: 
#   Input layer (784 dimensions)
#   layer 1 (1000 hidden neurons)
#   layer 2 (10 outputs)

import numpy
from keras.datasets import mnist
from keras.models import Sequential
from keras.layers import Dense
from keras.utils import np_utils

# load data
(X_train, y_train), (X_test, y_test) = mnist.load_data()

# flatten 28*28 images to a 784 vector for each image
num_pixels = X_train.shape[1] * X_train.shape[2]
X_train = X_train.reshape(X_train.shape[0], num_pixels).astype(&#39;float32&#39;)
X_test = X_test.reshape(X_test.shape[0], num_pixels).astype(&#39;float32&#39;)

# normalize inputs from 0-255 to 0-1
X_train = X_train / 255
X_test = X_test / 255

# one hot encode outputs
y_train = np_utils.to_categorical(y_train)
y_test = np_utils.to_categorical(y_test)
num_classes = y_test.shape[1]

# design model
model = Sequential()
model.add(Dense(1000, input_dim=num_pixels, activation=&#39;relu&#39;))
model.add(Dense(num_classes, activation=&#39;softmax&#39;))

# print out summary of the model
print(model.summary())

# Compile model
model.compile(loss=&#39;categorical_crossentropy&#39;, optimizer=&#39;adam&#39;, metrics=[&#39;accuracy&#39;])

# Fit the model
model.fit(X_train, y_train, validation_data=(X_test, y_test), epochs=10, batch_size=200, verbose=2)

# Final evaluation of the model
scores = model.evaluate(X_test, y_test, verbose=0)
print(&amp;quot;Baseline Error: %.2f%%&amp;quot; % (100-scores[1]*100))
&lt;/code&gt;&lt;/pre&gt;
&lt;hr&gt;
&lt;h3 id=&#34;2-assignment&#34;&gt;2. Assignment&lt;/h3&gt;
&lt;p&gt;Run the above code to double check your environment and get some sense about how it looks like while training a model. Design your own Neural Network to do Image Classification on &lt;strong&gt;Boat Dataset&lt;/strong&gt;. Boat Dataset consists of 5 different types of boats:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;
&lt;p&gt;Training Dataset (249.6 MB) &lt;a href=&#34;http://carina.cse.lehigh.edu/Data_Analytics_Course/resources/training_images.zip&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Download&lt;/a&gt;&lt;/p&gt;
&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th style=&#34;text-align:left&#34;&gt;Class&lt;/th&gt;
&lt;th style=&#34;text-align:left&#34;&gt;Number of images&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:left&#34;&gt;aircraft_carrier&lt;/td&gt;
&lt;td style=&#34;text-align:left&#34;&gt;500&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:left&#34;&gt;banana_boat&lt;/td&gt;
&lt;td style=&#34;text-align:left&#34;&gt;500&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:left&#34;&gt;oil_tanker&lt;/td&gt;
&lt;td style=&#34;text-align:left&#34;&gt;500&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:left&#34;&gt;passenger_ship&lt;/td&gt;
&lt;td style=&#34;text-align:left&#34;&gt;500&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:left&#34;&gt;yacht&lt;/td&gt;
&lt;td style=&#34;text-align:left&#34;&gt;500&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:left&#34;&gt;&lt;/td&gt;
&lt;td&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:left&#34;&gt;In total&lt;/td&gt;
&lt;td style=&#34;text-align:left&#34;&gt;2500&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Testing Dataset (97.1 MB, 1000 images) &lt;a href=&#34;http://carina.cse.lehigh.edu/Data_Analytics_Course/resources/testing_images.zip&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Download&lt;/a&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;Train your model on training dataset and test the trained model on testing dataset.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Hint&lt;/strong&gt;
Try to understand the following API(s):&lt;/p&gt;
&lt;pre&gt;&lt;code class=&#34;language-Python&#34;&gt;# All images in the new dataset are colorful images (consist of Red, Blue, and Green channels) with different sizes. You may would like to use the following API to resize all images into the same size before doing flatten (line 15 in the above code). 
image = cv2.imread(path) # load an image
resized_image = cv2.resize(image, (new_height, new_width)) # resize an image
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;To improve the performance of your model, you can try different values of the following parameters:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;input checked=&#34;&#34; disabled=&#34;&#34; type=&#34;checkbox&#34;&gt; number of hidden layers (network structure)&lt;/li&gt;
&lt;li&gt;&lt;input checked=&#34;&#34; disabled=&#34;&#34; type=&#34;checkbox&#34;&gt; number of neurons in each layer (network structure)&lt;/li&gt;
&lt;li&gt;&lt;input checked=&#34;&#34; disabled=&#34;&#34; type=&#34;checkbox&#34;&gt; image size (preprocessing)&lt;/li&gt;
&lt;li&gt;&lt;input checked=&#34;&#34; disabled=&#34;&#34; type=&#34;checkbox&#34;&gt; batch_size (training phase, model.compile)&lt;/li&gt;
&lt;li&gt;&lt;input checked=&#34;&#34; disabled=&#34;&#34; type=&#34;checkbox&#34;&gt; optimizer (training phase, model.compile)&lt;/li&gt;
&lt;li&gt;&lt;input checked=&#34;&#34; disabled=&#34;&#34; type=&#34;checkbox&#34;&gt; other settings you can dig out.&lt;/li&gt;
&lt;/ul&gt;
&lt;hr&gt;
&lt;h3 id=&#34;3-submite-your-sulotion&#34;&gt;3. Submite your sulotion:&lt;/h3&gt;
&lt;ul&gt;
&lt;li&gt;&lt;input disabled=&#34;&#34; type=&#34;checkbox&#34;&gt; Your final python code. Please name it using your Lehigh ID. (&amp;lt;your_LehighID&amp;gt;.py)&lt;/li&gt;
&lt;li&gt;&lt;input disabled=&#34;&#34; type=&#34;checkbox&#34;&gt; A short &amp;lt;your_LehighID&amp;gt;.pdf file. Simply describe what you did, what you got, and other things you want to report, e.g. what you have learned.&lt;/li&gt;
&lt;/ul&gt;
</description>
    </item>
    
    <item>
      <title></title>
      <link>http://xincoder.github.io/blogs/b_deep_learning/convolutional_network/</link>
      <pubDate>Fri, 18 May 2018 00:00:00 +0000</pubDate>
      <guid>http://xincoder.github.io/blogs/b_deep_learning/convolutional_network/</guid>
      <description>&lt;p&gt;In this blog, we are going to use Convolutional Neural Network (CNN) to do image classification.
The following figure shows the comparison between a 3-layer Neural Network and a simple Convolutional Neural Network. If you are interested in CNN, you can refer this paper which proposes &lt;a href=&#34;https://papers.nips.cc/paper/4824-imagenet-classification-with-deep-convolutional-neural-networks&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;AlexNet&lt;/a&gt;.
















&lt;figure  &gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img src=&#34;cnn.png&#34; alt=&#34;Screen Shot 2017-08-27 at 9.53.44 PM.png&#34; loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;/figure&gt;
&lt;/p&gt;
&lt;hr&gt;
&lt;h3 id=&#34;1-a-simple-convolutional-neural-network-on-mnist-datasethttpyannlecuncomexdbmnist-as-an-example&#34;&gt;1. A simple Convolutional Neural Network on &lt;a href=&#34;http://yann.lecun.com/exdb/mnist/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;MNIST dataset&lt;/a&gt; as an example.&lt;/h3&gt;
&lt;pre&gt;&lt;code class=&#34;language-Python&#34;&gt;import numpy
from keras.datasets import mnist
from keras.models import Sequential
from keras.layers import Dense, Dropout, Flatten
from keras.layers.convolutional import Conv2D, MaxPooling2D

# load data
(X_train, y_train), (X_test, y_test) = mnist.load_data()
# reshape to be [samples][pixels][width][height]
X_train = X_train.reshape(X_train.shape[0], 28, 28, 1).astype(&#39;float32&#39;)
X_test = X_test.reshape(X_test.shape[0], 28, 28, 1).astype(&#39;float32&#39;)

# normalize inputs from 0-255 to 0-1
X_train = X_train / 255
X_test = X_test / 255
# one hot encode outputs
y_train = np_utils.to_categorical(y_train)
y_test = np_utils.to_categorical(y_test)
num_classes = y_test.shape[1]

# create model
model = Sequential()
model.add(Conv2D(32, (5, 5), input_shape=(28, 28, 1), activation=&#39;relu&#39;))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.2))
model.add(Flatten())
model.add(Dense(128, activation=&#39;relu&#39;))
model.add(Dense(num_classes, activation=&#39;softmax&#39;))
# Compile model
model.compile(loss=&#39;categorical_crossentropy&#39;, optimizer=&#39;adam&#39;, metrics=[&#39;accuracy&#39;])

# Fit the model
model.fit(X_train, y_train, validation_data=(X_test, y_test), epochs=10, batch_size=200, verbose=2)

# Final evaluation of the model
scores = model.evaluate(X_test, y_test, verbose=0)
print(&amp;quot;Baseline Error: %.2f%%&amp;quot; % (100-scores[1]*100))	
&lt;/code&gt;&lt;/pre&gt;
&lt;hr&gt;
&lt;h3 id=&#34;2-assignment&#34;&gt;2. Assignment&lt;/h3&gt;
&lt;p&gt;Please run the above code before you design yours. You will notice that using a CNN model gains a higher accuracy than the Neural Netowork on MNIST dataset. Design your own CNN to do Image Classification on &lt;strong&gt;Boat Dataset&lt;/strong&gt;. Boat Dataset consists of 5 different types of boats:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Training Dataset (249.6 MB) &lt;a href=&#34;http://carina.cse.lehigh.edu/Data_Analytics_Course/resources/training_images.zip&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Download&lt;/a&gt;
&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th style=&#34;text-align:left&#34;&gt;Class&lt;/th&gt;
&lt;th style=&#34;text-align:left&#34;&gt;Number of images&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:left&#34;&gt;aircraft_carrier&lt;/td&gt;
&lt;td style=&#34;text-align:left&#34;&gt;500&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:left&#34;&gt;banana_boat&lt;/td&gt;
&lt;td style=&#34;text-align:left&#34;&gt;500&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:left&#34;&gt;oil_tanker&lt;/td&gt;
&lt;td style=&#34;text-align:left&#34;&gt;500&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:left&#34;&gt;passenger_ship&lt;/td&gt;
&lt;td style=&#34;text-align:left&#34;&gt;500&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:left&#34;&gt;yacht&lt;/td&gt;
&lt;td style=&#34;text-align:left&#34;&gt;500&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;/td&gt;
&lt;td&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:left&#34;&gt;In total&lt;/td&gt;
&lt;td style=&#34;text-align:left&#34;&gt;2500&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;
&lt;/li&gt;
&lt;li&gt;Testing Dataset (97.1 MB, 1000 images) &lt;a href=&#34;http://carina.cse.lehigh.edu/Data_Analytics_Course/resources/testing_images.zip&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Download&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;Train your model on training dataset and test the trained model on testing dataset.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Hint&lt;/strong&gt;&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;input disabled=&#34;&#34; type=&#34;checkbox&#34;&gt; Data augmentation (search how to do data augmentation in Keras)&lt;/li&gt;
&lt;li&gt;&lt;input disabled=&#34;&#34; type=&#34;checkbox&#34;&gt; Finetune a pre-trained CNN model. (refer to: keras.applications)&lt;/li&gt;
&lt;/ul&gt;
&lt;hr&gt;
&lt;h3 id=&#34;3-submite-your-sulotion&#34;&gt;3. Submite your sulotion:&lt;/h3&gt;
&lt;ul&gt;
&lt;li&gt;&lt;input disabled=&#34;&#34; type=&#34;checkbox&#34;&gt; Your final python code. Please name it using your Lehigh ID. (&amp;lt;your_LehighID&amp;gt;.py)&lt;/li&gt;
&lt;li&gt;&lt;input disabled=&#34;&#34; type=&#34;checkbox&#34;&gt; A short &amp;lt;your_LehighID&amp;gt;.pdf file. Simply describe what you did, what you got, and other things you want to report, e.g. what you have learned.&lt;/li&gt;
&lt;/ul&gt;
</description>
    </item>
    
    <item>
      <title></title>
      <link>http://xincoder.github.io/blogs/b_deep_learning/image_caption/</link>
      <pubDate>Fri, 18 May 2018 00:00:00 +0000</pubDate>
      <guid>http://xincoder.github.io/blogs/b_deep_learning/image_caption/</guid>
      <description>&lt;p&gt;In this blog, we are going to use LSTMs (Long Short Term Memory Networks) to generate a caption for a given image. LSTMs are a special kind of Recurrent Neural Networks (RNN). If you are looking for some related papers, please refer to &lt;a href=&#34;https://arxiv.org/pdf/1411.4555.pdf&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;paper1&lt;/a&gt; and &lt;a href=&#34;https://arxiv.org/pdf/1411.4389.pdf&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;paper2&lt;/a&gt;. The following figure shows the solution of image caption generation proposed in &lt;a href=&#34;https://arxiv.org/pdf/1411.4389.pdf&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;paper2&lt;/a&gt;.
&lt;img src=&#34;paper_figure.png&#34; alt=&#34;drawing&#34; width=&#34;500&#34;/&gt;&lt;/p&gt;
&lt;p&gt;Next, we will implement a simpler one. The following figure shows the architecture of the implemented model.&lt;/p&gt;
&lt;!-- ![model.png](quiver-image-url/65B111ACB5D56D0CCA6D1E63B7D3070E.png =950x2230) --&gt;
&lt;img src=&#34;model.png&#34; alt=&#34;drawing&#34; width=&#34;500&#34;/&gt;
___
### 1. Dataset
We use the [flickr30k](http://carina.cse.lehigh.edu/Data_Analytics_Course/resources/flickr30k_images.zip) dataset (4.39 GB) to train an image caption generator. The [flickr30k](http://carina.cse.lehigh.edu/Data_Analytics_Course/resources/flickr30k_images.zip) dataset consists of 31,783 images and each one has 5 corresponding captions. We split this dataset into a training subset (21,783 images) and a testing subset (10,000 images). 
&lt;hr&gt;
&lt;h3 id=&#34;2-a-simple-code&#34;&gt;2. A simple code&lt;/h3&gt;
&lt;p&gt;Please run the following command first:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&#34;language-Batch&#34;&gt;&amp;gt; pip install pillow
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Then, the following shows a simple demo code:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&#34;language-Python&#34;&gt;import os  
import numpy as np 
from keras.applications.vgg16 import VGG16, preprocess_input
from keras.models import Sequential, Model
from keras.layers import LSTM, Dense, Embedding, Merge, Flatten, RepeatVector, TimeDistributed, Concatenate
from keras.applications.vgg16 import VGG16, preprocess_input
from keras.preprocessing import image as Image
from keras.preprocessing import sequence as Sequence
from keras.callbacks import TensorBoard, ModelCheckpoint
from keras.utils import plot_model, to_categorical
from collections import Counter

CUDA_VISIBLE_DEVICES=&#39;0&#39;
os.environ[&amp;quot;CUDA_VISIBLE_DEVICES&amp;quot;] = CUDA_VISIBLE_DEVICES

# If you are running on your own computer, please change the following paths to your local paths.
# If you are running on HPC, you can keep the following paths.
# IMAGE_ROOT = &#39;/Users/xincoder/Documents/Dataset/flickr30k_images/flickr30k_images&#39;
# TRAIN_CAPTION_PATH = &#39;/Users/xincoder/Documents/Dataset/flickr30k_images/train.txt&#39;
# TEST_CAPTION_PATH = &#39;/Users/xincoder/Documents/Dataset/flickr30k_images/test.txt&#39;

IMAGE_ROOT = &#39;/share/ceph/mcc7cse498/xil915/flickr30k_images/flickr30k_images&#39;
TRAIN_CAPTION_PATH = &#39;/share/ceph/mcc7cse498/xil915/flickr30k_images/train.txt&#39;
TEST_CAPTION_PATH = &#39;/share/ceph/mcc7cse498/xil915/flickr30k_images/test.txt&#39;

WORDS_PATH = &#39;words.txt&#39;
SENTENCE_MAX_LENGTH = 100 # In this dataset, the maximum length is 84.
EMBEDDING_SIZE = 256
IMAGE_SIZE = 224

CHECK_ROOT = &#39;checkpoint/&#39;
if not os.path.exists(CHECK_ROOT):
	os.makedirs(CHECK_ROOT)

class Data_generator(object):
	def __init__(self, pra_batch_size=20, pra_word_frequency=2):
		self.word_frequency = pra_word_frequency # remove words whose frequency less than this value
		self.train_image_names, self.train_image_captions, self.test_image_names, self.test_image_captions = self.get_name_caption()
		self.train_image_captions_index = self.caption2index(self.train_image_captions)
		self.test_image_captions_index = self.caption2index(self.test_image_captions)
		self.batch_size = pra_batch_size # how many samples we want to train in each step
		self.train_steps_epoch = len(self.train_image_names)//pra_batch_size # steps per epoch
		self.test_steps_epoch = len(self.test_image_names)//pra_batch_size # steps per epoch
		
	def get_name_caption(self):
		&#39;&#39;&#39;
		Load training and testing data from files. 
		We add a &amp;lt;SOS&amp;gt; and &amp;lt;EOS&amp;gt; to the beginning and the end of each sentence respectively.
		(&amp;lt;SOS&amp;gt; stands for &amp;quot;start of sentence&amp;quot;, &amp;lt;EOS&amp;gt; stands for &amp;quot;end of sentence&amp;quot;)
		Returns:
			train_image_name_list: all paths of training images
			train_caption_list: corresponding training captions
			test_image_name_list: all paths of testing images
			test_caption_list: corresponding testing captions
		&#39;&#39;&#39;
		with open(TRAIN_CAPTION_PATH, &#39;r&#39;) as reader:
			content = [x.strip().split(&#39;\t&#39;) for x in reader.readlines()]
			train_image_name_list = [os.path.join(IMAGE_ROOT, x[0].split(&#39;#&#39;)[0]) for x in content]
			train_caption_list = [&#39;&amp;lt;SOS&amp;gt; {} &amp;lt;EOS&amp;gt;&#39;.format(x[1].lower()) for x in content]

		with open(TEST_CAPTION_PATH, &#39;r&#39;) as reader:
			content = [x.strip().split(&#39;\t&#39;) for x in reader.readlines()]
			test_image_name_list = [os.path.join(IMAGE_ROOT, x[0].split(&#39;#&#39;)[0]) for x in content]
			test_caption_list = [&#39;&amp;lt;SOS&amp;gt; {} &amp;lt;EOS&amp;gt;&#39;.format(x[1].lower()) for x in content]	

		all_words = &#39; &#39;.join(train_caption_list+test_caption_list).split(&#39; &#39;)
		words_num = Counter(all_words)
		words = [x for x in words_num if words_num[x]&amp;gt;=self.word_frequency]
		print(&#39;{} unique words (all).&#39;.format(len(words_num)))
		print(&#39;{} unique words (count&amp;gt;={}).&#39;.format(len(words), self.word_frequency))

		with open(WORDS_PATH, &#39;w&#39;) as writer:
			writer.write(&#39;\n&#39;.join(words))

		return train_image_name_list, train_caption_list, test_image_name_list, test_caption_list

	def get_dictionary(self, pra_captions):
		&#39;&#39;&#39; 
		Generate a dictionary for all words in our dataset. 
		Return:
			words2index: word-&amp;gt;index dictionary 
			index2words: index-&amp;gt;word dictionary
		&#39;&#39;&#39;
		if not os.path.exists(WORDS_PATH):
			words = set(&#39; &#39;.join(pra_captions).split(&#39; &#39;))
			with open(WORDS_PATH, &#39;w&#39;) as writer:
				writer.write(&#39;\n&#39;.join(words))
		else:
			with open(WORDS_PATH, &#39;r&#39;) as reader:
				words = [x.strip() for x in reader.readlines()]

		self.voc_size = len(words)
		words2index = dict((w, ind) for ind, w in enumerate(words, start=0))
		index2words = dict((ind, w) for ind, w in enumerate(words, start=0))
		return words2index, index2words

	def caption2index(self, pra_captions):
		words2index, index2words = self.get_dictionary(pra_captions)
		captions = [x.split(&#39; &#39;) for x in pra_captions]
		index_captions = [[words2index[w] for w in cap if w in words2index.keys()] for cap in captions]
		return index_captions

	def index2caption(self, pra_index):
		words2index, index2words = self.get_dictionary(&#39;&#39;)
		captions = [&#39; &#39;.join([index2words[w] for w in cap]) for cap in pra_index]
		return captions	

	def convert2onehot(self, pra_caption):
		captions = np.zeros((len(pra_caption), self.voc_size))
		for ind, cap in enumerate(pra_caption, start=0):
			captions[ind, cap] = 1
		return np.array(captions)

	def get_epoch_steps(self):
		return self.train_steps_epoch, self.test_steps_epoch

	def generate(self, pra_train=True):
		&#39;&#39;&#39;
		This is a generator which is used to continuously generate training or testing data.
			pra_train = True : generate training data
			pra_train = False : generate testing data
		&#39;&#39;&#39;
		while True:
			if pra_train:
				# we shuffle training data at the beginning of each epoch.
				shuffle_index = np.random.permutation(len(self.train_image_names))
				image_name_list = np.array(self.train_image_names)[shuffle_index]
				image_caption_list = np.array(self.train_image_captions)[shuffle_index]
				image_caption_index_list = np.array(self.train_image_captions_index)[shuffle_index]
			else:
				image_name_list = self.test_image_names
				image_caption_list = self.test_image_captions
				image_caption_index_list = self.test_image_captions_index

			image_caption_index_list = Sequence.pad_sequences(image_caption_index_list, maxlen=SENTENCE_MAX_LENGTH, padding=&#39;post&#39;)
			
			input_image_list = []
			input_caption_list = []
			target_caption_list = []
			for index, (image_name, image_caption) in enumerate(zip(image_name_list, image_caption_index_list), start=1):
				# image
				input_image = Image.img_to_array(Image.load_img(image_name, target_size=(IMAGE_SIZE, IMAGE_SIZE, 3)))
				input_caption_onehot = self.convert2onehot(image_caption)
				target_caption_onehot = np.zeros_like(input_caption_onehot)
				target_caption_onehot[:-1] = input_caption_onehot[1:]
				
				input_image_list.append(input_image)
				input_caption_list.append(input_caption_onehot)
				target_caption_list.append(target_caption_onehot)

				if len(input_image_list) == self.batch_size:
					tmp_images = np.array(input_image_list)
					tmp_captions = np.array(input_caption_list)
					tmp_targets = np.array(target_caption_list)
					input_image_list = []
					input_caption_list = []
					target_caption_list = []
					yield [preprocess_input(tmp_images), tmp_captions], tmp_targets


class Image_Caption(object):
	def __init__(self, pra_voc_size):
		self.voc_size = pra_voc_size

		# Model design start from here.
		# we use the VGG16 as the base model to extract CNN feature from an image
		base_model = VGG16(weights=&#39;imagenet&#39;, include_top=True)
		base_model = Model(inputs=base_model.input, outputs=base_model.get_layer(&#39;fc2&#39;).output)
		for layer in base_model.layers[1:]:
			layer.trainable = False

		# add a fully connected layer on the top of our base model 
		# and repeat it several times, so that it has the same shape as our language model
		image_model = Sequential()
		image_model.add(base_model)
		image_model.add(Dense(EMBEDDING_SIZE, activation=&#39;relu&#39;))
		image_model.add(RepeatVector(SENTENCE_MAX_LENGTH))
		
		# we use an Embedding layer to generate a good representation for captions.
		language_model = Sequential()
		# language_model.add(Embedding(self.voc_size, EMBEDDING_SIZE, input_length=SENTENCE_MAX_LENGTH))
		language_model.add(LSTM(128, input_shape=(SENTENCE_MAX_LENGTH, self.voc_size), return_sequences=True))
		language_model.add(TimeDistributed(Dense(128)))


		# after merging CNN feature (image) and embedded vector (caption), we feed them into a LSTM model
		# at its end, we use a fully connected layer with softmax activation to convert the output into probability 
		model = Sequential()
		model.add(Merge([image_model, language_model], mode=&#39;concat&#39;))
		# model.add(Concatenate([image_model, language_model]))
		model.add(LSTM(1000, return_sequences=True))
		# model.add(Dense(self.voc_size, activation=&#39;softmax&#39;, name=&#39;final_output&#39;))
		model.add(TimeDistributed(Dense(self.voc_size, activation=&#39;softmax&#39;)))

		# draw the model and save it to a file.
		# plot_model(model, to_file=&#39;model.pdf&#39;, show_shapes=True)
		
		self.model = model
		self.model.compile(loss=&#39;categorical_crossentropy&#39;, optimizer=&#39;rmsprop&#39;, metrics=[&#39;accuracy&#39;])

	def train_model(self, pra_datagen):
		# callback: draw curve on TensorBoard
		tensorboard = TensorBoard(log_dir=&#39;log&#39;, histogram_freq=0, write_graph=True, write_images=True)
		# callback: save the weight with the highest validation accuracy
		filepath=os.path.join(CHECK_ROOT, &#39;weights-improvement-{val_acc:.4f}-{epoch:04d}.hdf5&#39;)
		checkpoint = ModelCheckpoint(filepath, monitor=&#39;val_acc&#39;, verbose=2, save_best_only=True, mode=&#39;max&#39;)

		# train model 
		self.model.fit_generator(
			pra_datagen.generate(True), 
			steps_per_epoch=pra_datagen.get_epoch_steps()[0], 
			epochs=5, 
			validation_data=pra_datagen.generate(False), 
			validation_steps=pra_datagen.get_epoch_steps()[1],
			callbacks=[tensorboard, checkpoint])
	
if __name__ == &#39;__main__&#39;:
	my_generator = Data_generator()
	model = Image_Caption(my_generator.voc_size)
	model.train_model(my_generator)

	# for [img, cap], tar in my_generator.generate():
	# 	print(img.shape, cap.shape, tar.shape)
	# 	print(np.argmax(cap[0, 0]), np.argmax(tar[0, 0]))
	# 	print(np.argmax(cap[0, 1]), np.argmax(tar[0, 0]))
	# 	print(&#39;&#39;)
&lt;/code&gt;&lt;/pre&gt;
&lt;hr&gt;
&lt;h3 id=&#34;2-assignment&#34;&gt;2. Assignment&lt;/h3&gt;
&lt;ul&gt;
&lt;li&gt;&lt;input disabled=&#34;&#34; type=&#34;checkbox&#34;&gt; Run the above code to train the model and use it as a baseline.&lt;/li&gt;
&lt;li&gt;&lt;input disabled=&#34;&#34; type=&#34;checkbox&#34;&gt; Modify the above code to improve the accuracy.&lt;/li&gt;
&lt;li&gt;&lt;input disabled=&#34;&#34; type=&#34;checkbox&#34;&gt; Add a function to generate a sentence for a given image.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;&lt;strong&gt;Hint:&lt;/strong&gt;
To improve the accuracy, you can try:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;different base models. (refer to: keras.applications)&lt;/li&gt;
&lt;li&gt;different RNNs. (refer to: keras.layers.recurrent)&lt;/li&gt;
&lt;li&gt;data augmentation by randomly rotating, fliping, or shifting training images.&lt;/li&gt;
&lt;/ul&gt;
&lt;hr&gt;
&lt;h3 id=&#34;3-submite-your-sulotion&#34;&gt;3. Submite your sulotion:&lt;/h3&gt;
&lt;ul&gt;
&lt;li&gt;&lt;input disabled=&#34;&#34; type=&#34;checkbox&#34;&gt; Your final python code. Please name it using your Lehigh ID. (&amp;lt;your_LehighID&amp;gt;.py)&lt;/li&gt;
&lt;li&gt;&lt;input disabled=&#34;&#34; type=&#34;checkbox&#34;&gt; A short &amp;lt;your_LehighID&amp;gt;.pdf file. Simply describe what you did, what you got, and other things you want to report, e.g. what you have learned.&lt;/li&gt;
&lt;/ul&gt;
&lt;hr&gt;
&lt;h3 id=&#34;4-train-your-model-on-hpc&#34;&gt;4. Train your model on HPC&lt;/h3&gt;
&lt;p&gt;It will take weeks (or months) if you only use the CPU on your laptops to train the model. Considering this, Prof. Chuah has already applied Lehigh University Research Computing (HPC) resource for all of you. (You may have already received an email from &lt;a href=&#34;root@polaris.cc.lehigh.edu&#34;&gt;root@polaris.cc.lehigh.edu&lt;/a&gt;). Please run your code on your own computer first to make sure that there is no error before you run it on HPC.&lt;/p&gt;
&lt;p&gt;You can access HPC via SSH.&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;For Windows users: please download &lt;a href=&#34;http://www.putty.org/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Putty&lt;/a&gt;.&lt;/li&gt;
&lt;li&gt;For Mac users: you can use SSH in a terminal.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;The username and password for HPC is your LehighID and the corresponding password. For example, my LehighID id &lt;strong&gt;xil915&lt;/strong&gt;, then I can access HPC using the following command:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&#34;language-Bash&#34;&gt;ssh &amp;lt;your_LehighID&amp;gt;@sol.cc.lehigh.edu
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;All training and testing data have been saved in a shared directory:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&#34;language-Bash&#34;&gt;/share/ceph/.../.../flickr30k_images
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Once you login, you need to create two files in your own directories:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;your python code, namely image_caption.py.&lt;/li&gt;
&lt;li&gt;a bash file, namely run.sh&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;Save the following script into your run.sh:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&#34;language-Bash&#34;&gt;#!/bin/tcsh
#SBATCH --partition=imlab-gpu 
#SBATCH --time=100:00:00 # maximum time
#SBATCH --nodes=1 # 1 CPU can be be paired with only 1 GPU
#SBATCH --ntasks-per-node=1
#SBATCH --gres=gpu:1 # Need both GPUs, use --gres=gpu:2
#SBATCH --job-name xin_image_caption
#SBATCH --output=&amp;quot;log.%j.%N.out&amp;quot;

module load python/cse498
python image_caption.py
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Run your code:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&#34;language-Bash&#34;&gt;sbatch run.sh
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;This command will sumbit your job to a waitinglist. Please use the following command to check the status of your jobs:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&#34;language-Bash&#34;&gt;&amp;gt; squeue # list all jobs
&amp;gt; squeue -u xil915 # your LehighID, only list your job(s).
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;This is my output:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&#34;language-Bash&#34;&gt;[xil915@sol CSE498]$ squeue -u xil915
    JOBID PARTITION     NAME     USER ST       TIME  NODES NODELIST(REASON)
    187364 imlab-gpu xin_imag   xil915 PD       0:00      1 (Priority)
# ST == PD : this job is pending
# ST == R : this job is running
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Cancel your job using:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&#34;language-Bash&#34;&gt;&amp;gt; scancel 187364 # JOBID
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;When your job is running, all standard outputs will be saved in to file namely, log.*.out (e.g. log.187364.sol-b411.out). You can print it out using:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&#34;language-Bash&#34;&gt;&amp;gt; cat log.187364.sol-b411.out
&lt;/code&gt;&lt;/pre&gt;
&lt;hr&gt;
&lt;h3 id=&#34;span-stylecolororange-help-span&#34;&gt;&lt;span style=&#34;color:orange&#34;&gt; HELP !!!&lt;/span&gt;&lt;/h3&gt;
&lt;p&gt;Considering that the long waiting list on HPC, I provide a pre-trained model using my demo code to you.&lt;/p&gt;
&lt;p&gt;During your model is training, you can use my pre-trained model to test your implemented function. (generating a sentence for a given image)&lt;/p&gt;
&lt;p&gt;Downloads: &lt;a href=&#34;http://carina.cse.lehigh.edu/Data_Analytics_Course/resources/xin_weights.hdf5&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;pre-trained model&lt;/a&gt;, &lt;a href=&#34;http://carina.cse.lehigh.edu/Data_Analytics_Course/resources/words.txt&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;words index&lt;/a&gt;&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;The pre-trained model only was trained for 2 epochs.&lt;/li&gt;
&lt;li&gt;The words index is a file which list the indices of words.
(&lt;span style=&#34;color:red&#34;&gt; A good result is not expected using this pre-trained model, considering that this is only trained for 2 epochs.&lt;/span&gt;)&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;Load weights:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&#34;language-python&#34;&gt;model.load_weights(filepath)
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;&lt;span style=&#34;color:orange&#34;&gt;Please note it in your report, if you use this pre-trained model to generate the final results.&lt;/span&gt;&lt;/p&gt;
</description>
    </item>
    
    <item>
      <title></title>
      <link>http://xincoder.github.io/blogs/b_deep_learning/train_classifier/</link>
      <pubDate>Fri, 18 May 2018 00:00:00 +0000</pubDate>
      <guid>http://xincoder.github.io/blogs/b_deep_learning/train_classifier/</guid>
      <description>&lt;h3 id=&#34;how-to-train-an-object-classifier-using-our-own-images&#34;&gt;How to train an object classifier using our own images&lt;/h3&gt;
&lt;h3 id=&#34;1-info&#34;&gt;1. Info:&lt;/h3&gt;
&lt;p&gt;I prepared two python scripts (&lt;a href=&#34;retrain.py&#34;&gt;retrain.py&lt;/a&gt;, &lt;a href=&#34;predict.py&#34;&gt;predict.py&lt;/a&gt;) for this task.&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;a href=&#34;retrain.py&#34;&gt;retrain.py&lt;/a&gt;: used to train the classifier.&lt;/li&gt;
&lt;li&gt;&lt;a href=&#34;predict.py&#34;&gt;predict.py&lt;/a&gt;: used to load the trained model and test on new images.&lt;/li&gt;
&lt;/ul&gt;
&lt;hr&gt;
&lt;h3 id=&#34;2-prepare-training-and-testing-data&#34;&gt;2. Prepare training and testing data:&lt;/h3&gt;
&lt;ul&gt;
&lt;li&gt;
&lt;p&gt;&lt;strong&gt;Training Data:&lt;/strong&gt; Let&amp;rsquo;s assume that we have two classes, namely &amp;ldquo;cat&amp;rdquo; and &amp;ldquo;dog&amp;rdquo;. We just need to make sure that there are two sub-folders in &amp;ldquo;training_images&amp;rdquo; folder. Each sub-folder only consists of its own images.&lt;/p&gt;
&lt;pre&gt;&lt;code class=&#34;language-plaintext&#34;&gt;|- training_images
    |- cat
        |- image_0.jpg
        |- image_1.jpg
        ...
    |- dog
        |- image_2.jpg
        |- image_3.jpg
        ...
&lt;/code&gt;&lt;/pre&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;&lt;strong&gt;Testing Data:&lt;/strong&gt; All testing images are put in one folder, e.g. &amp;ldquo;testing_images&amp;rdquo;.&lt;/p&gt;
&lt;pre&gt;&lt;code class=&#34;language-plaintext&#34;&gt;|- testing_images
    |- image_0.jpg
    |- image_1.jpg
    |- image_2.jpg
    |- image_3.jpg
    ...
&lt;/code&gt;&lt;/pre&gt;
&lt;/li&gt;
&lt;/ul&gt;
&lt;hr&gt;
&lt;h3 id=&#34;3-train-a-classifier&#34;&gt;3. Train a classifier&lt;/h3&gt;
&lt;p&gt;Assume that both &amp;ldquo;training_images&amp;rdquo; and &amp;ldquo;testing_images&amp;rdquo; are in this folder:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&#34;language-bash&#34;&gt;&amp;gt; ls
models/ predict.py  retrain.py  testing_images/  training_images/
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Then, we can type the following command to starting training process:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&#34;language-bash&#34;&gt;&amp;gt; python retrain.py \
--bottleneck_dir=./bottlenecks \
--how_many_training_steps=50000 \ 
--model_dir=./inception \
--output_graph=./models/retrained_graph.pb \
--output_labels=./models/retrained_labels.txt \
--summaries_dir=./retrain_logs \
--validation_batch_size=5000 \
--image_dir=training_images # this is the folder of training data
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;After this training process finished, it saves the trained model in &amp;ldquo;./models&amp;rdquo;.&lt;/p&gt;
&lt;pre&gt;&lt;code class=&#34;language-bash&#34;&gt;&amp;gt; ls models
retrained_graph.pb    retrained_labels.txt
&lt;/code&gt;&lt;/pre&gt;
&lt;hr&gt;
&lt;h3 id=&#34;4-predict-new-images-using-trained-model&#34;&gt;4. Predict new images using trained model&lt;/h3&gt;
&lt;p&gt;Type the following command in a terminal to run the testing code:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&#34;language-Bash&#34;&gt;&amp;gt; python predict.py \
--models_folder=&#39;./models&#39; \
--test_image_folder=&#39;./testing_images&#39; \
--display_image=False
&lt;/code&gt;&lt;/pre&gt;
</description>
    </item>
    
    <item>
      <title></title>
      <link>http://xincoder.github.io/blogs/b_deep_learning/model_on_android/</link>
      <pubDate>Fri, 18 May 2018 00:00:00 +0000</pubDate>
      <guid>http://xincoder.github.io/blogs/b_deep_learning/model_on_android/</guid>
      <description>&lt;h3 id=&#34;how-to-use-a-pre-trained-mode-on-an-android-device&#34;&gt;How to use a pre-trained mode on an Android device&lt;/h3&gt;
&lt;p&gt;In previous chapter, we discussed how to train an object classifier using our own images. At the end, we got trained model and labels file (retrained_graph.pb, retrained_labels.txt).&lt;/p&gt;
&lt;p&gt;In this chapter, We are going to load pre-trained classifer in our Android app. Unfortunately, we can not use the trained model on Android directly. We need to optimize it using a tool, namely &amp;ldquo;optimize_for_inference&amp;rdquo;, provided by Tensorflow.&lt;/p&gt;
&lt;hr&gt;
&lt;h3 id=&#34;1-build-tool-optimize_for_inference&#34;&gt;1. Build tool &amp;ldquo;optimize_for_inference&amp;rdquo;&lt;/h3&gt;
&lt;ul&gt;
&lt;li&gt;
&lt;p&gt;Download Tensorflow source code:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&#34;language-bash&#34;&gt;git clone https://github.com/tensorflow/tensorflow.git
&lt;/code&gt;&lt;/pre&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Install bazel so that we can use it to build &amp;ldquo;optimize_for_inference&amp;rdquo;&lt;/p&gt;
&lt;pre&gt;&lt;code class=&#34;language-bash&#34;&gt;echo &amp;quot;deb [arch=amd64] http://storage.googleapis.com/bazel-apt stable jdk1.8&amp;quot; | sudo tee /etc/apt/sources.list.d/bazel.list
curl https://bazel.build/bazel-release.pub.gpg | sudo apt-key add -
sudo apt-get update &amp;amp;&amp;amp; sudo apt-get install bazel
sudo apt-get upgrade bazel
&lt;/code&gt;&lt;/pre&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Build &amp;ldquo;optimize_for_inference&amp;rdquo;&lt;/p&gt;
&lt;pre&gt;&lt;code class=&#34;language-bash&#34;&gt;cd tensorflow
./configure # We can choose all default settings
bazel build tensorflow/python/tools:optimize_for_inference # this process takes a while, be patient
&lt;/code&gt;&lt;/pre&gt;
&lt;/li&gt;
&lt;/ul&gt;
&lt;hr&gt;
&lt;h3 id=&#34;2-optimize-trained-model&#34;&gt;2. Optimize trained model&lt;/h3&gt;
&lt;p&gt;Let&amp;rsquo;s assume that our pre-trained model is &lt;strong&gt;&amp;lt;folder_path&amp;gt;/retrained_graph.pb&lt;/strong&gt;. Then, we can use the following command to optimize the model and save it as &lt;strong&gt;&amp;lt;folder_path&amp;gt;/retrained_graph_android.pb&lt;/strong&gt;&lt;/p&gt;
&lt;pre&gt;&lt;code class=&#34;language-bash&#34;&gt;bazel-bin/tensorflow/python/tools/optimize_for_inference \
--input=&amp;lt;folder_path&amp;gt;/retrained_graph.pb \
--output=&amp;lt;folder_path&amp;gt;/retrained_graph_android.pb \
--input_names=Mul \
--output_names=final_result
&lt;/code&gt;&lt;/pre&gt;
&lt;hr&gt;
&lt;h3 id=&#34;3-modify-tensorflow-android-demo&#34;&gt;3. Modify Tensorflow Android Demo&lt;/h3&gt;
&lt;ul&gt;
&lt;li&gt;
&lt;p&gt;Download Android Demo:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&#34;language-bash&#34;&gt;git clone https://github.com/Nilhcem/tensorflow-classifier-android.git
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;If we import this project into Android Studio, compile and run, the demo will load a pre-trained classifier which can recognize 1000 classes.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Load our own model:&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;Delete the previous ImageNet model from &lt;strong&gt;assets/&lt;/strong&gt; folder.&lt;/li&gt;
&lt;li&gt;Copy our optimized trained model &lt;strong&gt;retrained_graph_android.pb&lt;/strong&gt; and label file &lt;strong&gt;retrained_labels.txt&lt;/strong&gt; into &lt;strong&gt;assets/&lt;/strong&gt; folder.&lt;/li&gt;
&lt;li&gt;Open &lt;strong&gt;ClassifierActivity.java&lt;/strong&gt; and set the following variables:&lt;/li&gt;
&lt;/ol&gt;
&lt;pre&gt;&lt;code class=&#34;language-Java&#34;&gt;  private static final int INPUT_SIZE = 299; 
  private static final int IMAGE_MEAN = 128; 
  private static final float IMAGE_STD = 128; 
  private static final String INPUT_NAME = &amp;quot;Mul&amp;quot;;
  private static final String OUTPUT_NAME = &amp;quot;final_result&amp;quot;;
&lt;/code&gt;&lt;/pre&gt;
&lt;ol start=&#34;4&#34;&gt;
&lt;li&gt;Compile and run. The demo will open the camera and show the confidence score of each corresponding class.&lt;/li&gt;
&lt;/ol&gt;
&lt;/li&gt;
&lt;/ul&gt;
</description>
    </item>
    
    <item>
      <title></title>
      <link>http://xincoder.github.io/blogs/b_deep_learning/boat_classifier/</link>
      <pubDate>Fri, 18 May 2018 00:00:00 +0000</pubDate>
      <guid>http://xincoder.github.io/blogs/b_deep_learning/boat_classifier/</guid>
      <description>&lt;p&gt;In this chapter, I list all materials related to the trained classifier which is used to classify an image into 5 different types of boats:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;iceboat&lt;/li&gt;
&lt;li&gt;shrimper&lt;/li&gt;
&lt;li&gt;patrol boat&lt;/li&gt;
&lt;li&gt;fishing boat&lt;/li&gt;
&lt;li&gt;weather ship&lt;/li&gt;
&lt;/ul&gt;
&lt;hr&gt;
&lt;h3 id=&#34;0-trained-model&#34;&gt;0. Trained model&lt;/h3&gt;
&lt;p&gt;The trained model consists of two files:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;a href=&#34;http://carina.cse.lehigh.edu/Xincoder_readme/resources/retrained_graph.pb&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;retrained_graph.pb&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href=&#34;http://carina.cse.lehigh.edu/Xincoder_readme/resources/retrained_labels.txt&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;retrained_labels.txt&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;
&lt;h3 id=&#34;1-downloaded-dataset&#34;&gt;1. Downloaded Dataset&lt;/h3&gt;
&lt;ul&gt;
&lt;li&gt;Training dataset (475.6 MB) &lt;a href=&#34;http://carina.cse.lehigh.edu/Xincoder_readme/resources/training_images.zip&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Download&lt;/a&gt;
&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th style=&#34;text-align:center&#34;&gt;&lt;strong&gt;Class&lt;/strong&gt;&lt;/th&gt;
&lt;th style=&#34;text-align:center&#34;&gt;&lt;strong&gt;Number of images&lt;/strong&gt;&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;iceboat&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;1692&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;shrimper&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;2380&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;patrol_boat&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;1820&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;fishing_boat&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;1320&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;weather_ship&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;1101&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;In total&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;8313&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;
&lt;/li&gt;
&lt;li&gt;Testing dataset (34.7 MB, 1532 images) &lt;a href=&#34;http://carina.cse.lehigh.edu/Xincoder_readme/resources/testing_images.zip&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Download&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;
&lt;h3 id=&#34;2-test-trained-model&#34;&gt;2. Test trained model&lt;/h3&gt;
&lt;p&gt;After we downloaded the trained model and Testing dataset, can use &lt;a href=&#34;../train_classifier/predict.py&#34;&gt;this code&lt;/a&gt; to test this model by type the following command in a terminal:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&#34;language-Bash&#34;&gt;&amp;gt; python predict.py \
--models_folder=&#39;./models&#39; \
--test_image_folder=&#39;./testing_images&#39; \
--display_image=False
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;















&lt;figure  &gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img src=&#34;fig_probability.png&#34; alt=&#34;&#34; loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;/figure&gt;
&lt;/p&gt;
&lt;h3 id=&#34;3-use-trained-model-on-android&#34;&gt;3. Use trained model on Android&lt;/h3&gt;
&lt;p&gt;This android demo loads our trained boat classifier and classify the camera video frame.&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Android apk file (103.4 MB): &lt;a href=&#34;http://carina.cse.lehigh.edu/Xincoder_readme/resources/Xin_Boat_Demo.apk&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Xin_Boat_Demo.apk&lt;/a&gt; (I only tested this app on our ASUS NEXUS 7 Tablet.)&lt;/li&gt;
&lt;li&gt;Source Code (392.1 MB): &lt;a href=&#34;http://carina.cse.lehigh.edu/Xincoder_readme/resources/tensorflow-classifier-android.zip&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;tensorflow-classifier-android.zip&lt;/a&gt; (already contained trained model)&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;This figure is the screenshot:
















&lt;figure  &gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img src=&#34;demo.png&#34; alt=&#34;&#34; loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;/figure&gt;
&lt;/p&gt;
&lt;hr&gt;
&lt;hr&gt;
&lt;h3 id=&#34;if-you-do-not-want-to-know-more-details-about-how-did-i-collect-the-data-you-can-skip-the-following-part-of-this-chapter&#34;&gt;If you do not want to know more details about how did I collect the data, you can skip the following part of this chapter.&lt;/h3&gt;
&lt;hr&gt;
&lt;hr&gt;
&lt;h3 id=&#34;image-collection&#34;&gt;Image Collection&lt;/h3&gt;
&lt;ol&gt;
&lt;li&gt;All training images are downloaded from &lt;a href=&#34;http://www.image-net.org/&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;ImageNet&lt;/a&gt;. In order to make it easier, I wrote a python script (&lt;a href=&#34;http://carina.cse.lehigh.edu/Xincoder_readme/resources/download_image.py&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;download_image.py&lt;/a&gt;) to download images automatically. (PS: some time the it takes a long time to access ImageNet, which causes a time out error of the script. We need to login the ImageNet (easy to register an account) in our browser first.)&lt;/li&gt;
&lt;li&gt;There are some links not accessable any more, which results in some downloaded files are not images. Thus, I wrote another python script (&lt;a href=&#34;http://carina.cse.lehigh.edu/Xincoder_readme/resources/remove.py&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;remove.py&lt;/a&gt;) to remove them. In addition, some images are error message and we need to manually remove them.&lt;/li&gt;
&lt;li&gt;I used &lt;a href=&#34;https://github.com/codebox/image_augmentor&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;this tool&lt;/a&gt; to augment the training images.&lt;/li&gt;
&lt;/ol&gt;
</description>
    </item>
    
    <item>
      <title></title>
      <link>http://xincoder.github.io/blogs/a_server_setup/panda_list/</link>
      <pubDate>Wed, 16 May 2018 00:00:00 +0000</pubDate>
      <guid>http://xincoder.github.io/blogs/a_server_setup/panda_list/</guid>
      <description>&lt;p&gt;I built a server with 2 Nvidia Titan Xp for my research. The following table shows the details of my own sever:&lt;/p&gt;
&lt;hr&gt;
&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th style=&#34;text-align:center&#34;&gt;Hardware&lt;/th&gt;
&lt;th&gt;Detail&lt;/th&gt;
&lt;th style=&#34;text-align:center&#34;&gt;Quantity&lt;/th&gt;
&lt;th style=&#34;text-align:right&#34;&gt;Price&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;CPU&lt;/td&gt;
&lt;td&gt;Intel - Xeon E5-2630 V4 2.2GHz 10-Core Processor&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;1&lt;/td&gt;
&lt;td style=&#34;text-align:right&#34;&gt;$679.99&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;Liquid CPU Cooler&lt;/td&gt;
&lt;td&gt;NZXT - Kraken X62 Liquid CPU Cooler&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;1&lt;/td&gt;
&lt;td style=&#34;text-align:right&#34;&gt;$156.29&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;Memory&lt;/td&gt;
&lt;td&gt;Corsair - Vengeance LPX 32GB (2 x 16GB) DDR4-2400 Memory&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;2&lt;/td&gt;
&lt;td style=&#34;text-align:right&#34;&gt;$347.02&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;MotherBoard&lt;/td&gt;
&lt;td&gt;Asus - ROG STRIX X99 GAMING ATX LGA2011-3 Motherboard&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;1&lt;/td&gt;
&lt;td style=&#34;text-align:right&#34;&gt;$306.74&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;Graphics Card&lt;/td&gt;
&lt;td&gt;NVIDIA GeForce Titan X Pascal 12GB GDDR5X&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;2&lt;/td&gt;
&lt;td style=&#34;text-align:right&#34;&gt;$2,400.00&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;SLI Bridge&lt;/td&gt;
&lt;td&gt;ASUS ROG SLI High-Bandwidth Bridge with Aura Sync RGB 3 Slot (ROG-SLI-HB-BRIDGE-3SLOT)&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;1&lt;/td&gt;
&lt;td style=&#34;text-align:right&#34;&gt;$49.99&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;SSD&lt;/td&gt;
&lt;td&gt;Intel 535 Series 240GB SATA III 6Gb/s 2.5 Solid State Drive&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;2&lt;/td&gt;
&lt;td style=&#34;text-align:right&#34;&gt;$69.99&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;SSD&lt;/td&gt;
&lt;td&gt;Samsung 850 EVO 1TB 2.5-Inch SATA III Internal SSD (MZ-75E1T0B/AM)&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;2&lt;/td&gt;
&lt;td style=&#34;text-align:right&#34;&gt;$599.98&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;Power Supply&lt;/td&gt;
&lt;td&gt;Corsair HXi Series, HX1000i 1000 Watt Fully Modular Power Supply&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;1&lt;/td&gt;
&lt;td style=&#34;text-align:right&#34;&gt;$219.99&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;Case&lt;/td&gt;
&lt;td&gt;Phanteks - Enthoo Evolv ATX ATX Mid Tower Case&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;1&lt;/td&gt;
&lt;td style=&#34;text-align:right&#34;&gt;$175.29&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;Keyboard &amp;amp; Mouse&lt;/td&gt;
&lt;td&gt;Microsoft Wireless Desktop 900 Keyboard &amp;amp; Mouse Combo&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;1&lt;/td&gt;
&lt;td style=&#34;text-align:right&#34;&gt;$29.99&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;Monitor&lt;/td&gt;
&lt;td&gt;Dell UltraSharp U2414H 24-Inch Screen LED Monitor&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;1&lt;/td&gt;
&lt;td style=&#34;text-align:right&#34;&gt;$214.72&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;Wireless Adapter&lt;/td&gt;
&lt;td&gt;Panda Wireless PAU06 300Mbps N USB Adapter&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;1&lt;/td&gt;
&lt;td style=&#34;text-align:right&#34;&gt;$19.99&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;UPS&lt;/td&gt;
&lt;td&gt;APC - SMT1500 UPS&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;1&lt;/td&gt;
&lt;td style=&#34;text-align:right&#34;&gt;$419.00&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;&lt;/td&gt;
&lt;td&gt;&lt;/td&gt;
&lt;td&gt;&lt;/td&gt;
&lt;td&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;Before Tax:&lt;/td&gt;
&lt;td&gt;&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;&lt;/td&gt;
&lt;td style=&#34;text-align:right&#34;&gt;$5,688.98&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;Tax:&lt;/td&gt;
&lt;td&gt;&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;&lt;/td&gt;
&lt;td style=&#34;text-align:right&#34;&gt;$341.34&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;In Total:&lt;/td&gt;
&lt;td&gt;&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;&lt;/td&gt;
&lt;td style=&#34;text-align:right&#34;&gt;$6,030.32&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;
&lt;hr&gt;
</description>
    </item>
    
    <item>
      <title></title>
      <link>http://xincoder.github.io/blogs/a_server_setup/setup_environment/</link>
      <pubDate>Wed, 16 May 2018 00:00:00 +0000</pubDate>
      <guid>http://xincoder.github.io/blogs/a_server_setup/setup_environment/</guid>
      <description>&lt;p&gt;After building a sever, I did the following operations to setup the environment.&lt;/p&gt;
&lt;h3 id=&#34;1-create-a-usb-ubuntu-installer-on-mac&#34;&gt;1. Create a USB ubuntu Installer (on Mac)&lt;/h3&gt;
&lt;p&gt;Create a bootable USB on MacOS, so that we can use it to install &lt;a href=&#34;https://www.ubuntu.com/download/desktop&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Ubuntu 16.04&lt;/a&gt;.&lt;/p&gt;
&lt;pre&gt;&lt;code class=&#34;language-Bash&#34;&gt;# Eject the USB from your MAC
cd ~/Downloads
# rename the ISO file with a shorter name
hdiutil convert -format UDRW -o ubuntu.iso  ubuntu-16.04.3-desktop-amd64.iso
mv ubuntu.iso.dmg ubuntu.iso
diskutil list

# Plug in the USB and figure out the disk ID.
diskutil list
diskutil unmountDisk /dev/disk3 # disk3 is my USB
sudo dd if=./ubuntu.iso of=/dev/rdisk3 bs=1m # disk3 is my USB
&lt;/code&gt;&lt;/pre&gt;
&lt;hr&gt;
&lt;h3 id=&#34;2-install-cuda&#34;&gt;2. Install CUDA&lt;/h3&gt;
&lt;p&gt;Download Nvidia display driver from &lt;a href=&#34;https://www.geforce.com/drivers&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Nvidia&lt;/a&gt;. (NVIDIA-Linux_*.run)
Download CUDA &amp;ldquo;runfile&amp;rdquo; from &lt;a href=&#34;https://developer.nvidia.com/cuda-downloads&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Nvidia&lt;/a&gt;.&lt;/p&gt;
&lt;pre&gt;&lt;code class=&#34;language-Bash&#34;&gt;# Install SSH server
sudo apt-get install openssh-server

# Turn off the lightdm 
sudo service lightdm stop
sudo apt-get install vim
sudo apt-get install dkms build-essential linux-headers-generic
sudo vi /etc/modprobe.d/blacklist.conf
# insert the following lines to the end:
blacklist nouveau
#blacklist lbm-nouveau
#options nouveau modeset=0
#alias nouveau off
#alias lbm-nouveau off

#&amp;gt; echo options nouveau modeset=0 | sudo tee -a /etc/modprobe.d/nouveau-kms.conf
sudo update-initramfs -u

sudo reboot
sudo service lightdm stop
sudo ./NVIDIA-Linux-x86_64-390.48.run --no-x-check --no-nouveau-check --no-opengl-files
sudo reboot
sudo ./cuda_9.0.176_384.81_linux.run --no-opengl-libs
# This time, do not install driver, do not choose opengl and X configuration

vi ~/.bashrc
# insert the following lines to the end:
export LD_LIBRARY_PATH=&amp;quot;/usr/local/cuda-8.0/lib64/&amp;quot;
export CUDA_BIN=/usr/local/cuda-8.0/bin
export CUDA_LIB=/usr/local/cuda-8.0/lib64
export PATH=${CUDA_BIN}:$PATH

# &amp;gt; sudo reboot
&lt;/code&gt;&lt;/pre&gt;
&lt;hr&gt;
&lt;h3 id=&#34;3-enable-nvidia-driver&#34;&gt;3. Enable Nvidia Driver&lt;/h3&gt;
&lt;p&gt;After reboot, open &lt;strong&gt;&amp;ldquo;Additional Drivers&amp;rdquo;&lt;/strong&gt; and choose &amp;ldquo;Using Nvidia binary driver&amp;rdquo;.&lt;/p&gt;
&lt;hr&gt;
&lt;h3 id=&#34;4-install-cudnn&#34;&gt;4. Install CuDNN&lt;/h3&gt;
&lt;p&gt;Download &lt;a href=&#34;https://developer.nvidia.com/cudnn&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;CuDNN 6.0&lt;/a&gt;.&lt;/p&gt;
&lt;pre&gt;&lt;code class=&#34;language-Bash&#34;&gt;tar -zxvf cudnn-8.0-linux-x64-v6.0tgz
cd cuda
sudo cp lib* /usr/local/cuda-8.0/lib64/
sudo cp cudnn.h /usr/local/cuda-8.0/include/
cd /usr/local/cuda-8.0/lib64
# update links
sudo rm libcudnn.so libcudnn.so.6
sudo ln -s libcudnn.so.6.0.21 libcudnn.so.6
sudo ln -s libcudnn.so.6 libcudnn.so
&lt;/code&gt;&lt;/pre&gt;
&lt;hr&gt;
&lt;h3 id=&#34;5-install-keras&#34;&gt;5. Install Keras&lt;/h3&gt;
&lt;pre&gt;&lt;code class=&#34;language-Bash&#34;&gt;# Step 1: Download Anaconda
# On Linux
wget https://repo.continuum.io/archive/Anaconda3-4.4.0-Linux-x86_64.sh
# On MacOS
curl -O https://repo.continuum.io/archive/Anaconda3-4.4.0-MacOSX-x86_64.sh

# Step 2: Install Anaconda (Use all default settings)
bash Anaconda3-4.4.0-MacOSX-x86_64.sh

# Step 3: Restart your terminal

# Step 4: Create a virtual environment. (so that it will not mess up the existing settings) 
conda create -n keras python=3.5

# Step 5: Install Tensorflow 
source activate keras
# GPU version on Linux
pip install tensorflow-gpu
# CPU version on Mac
pip install --upgrade https://storage.googleapis.com/tensorflow/mac/cpu/tensorflow-1.3.0-py3-none-any.whl
# If there is an error, please try it again.

# Step 6: Install Keras
# on MacOs
brew install graphviz 
# On Linux
sudo apt-get install python-pydot python-pydot-ng graphviz 

pip install keras

# Step 7: Install other Dependencies
conda install HDF5
conda install h5py
pip install pydot
pip install graphviz
pip install pillow
pip install opencv-python
# conda install -c https://conda.anaconda.org/menpo opencv3
# for visualize the model 
pip install quiver_engine 
pip install keras-vis

# Step 8: Test
python
&lt;/code&gt;&lt;/pre&gt;
&lt;pre&gt;&lt;code class=&#34;language-Python&#34;&gt;from keras.models import Sequential
from keras.layers import Dense
from keras.utils import plot_model
model = Sequential()
model.add(Dense(10, input_shape=(700, 1)))
model.summary()
plot_model(model, to_file=&#39;abc.pdf&#39;, show_shapes=True)
exit()
&lt;/code&gt;&lt;/pre&gt;
&lt;hr&gt;
&lt;h3 id=&#34;6-install-other-dependencies&#34;&gt;6. Install other dependencies&lt;/h3&gt;
&lt;pre&gt;&lt;code class=&#34;language-Bash&#34;&gt;# moviepy
pip install moviepy
# If there is a ffmpeg error, please add the following lines at the top of your python file:
    import imageio
    imageio.plugins.ffmpeg.download()

# ERROR
# This error can ben due to the fact that ImageMagick is not installed on your computer. 
# on ubuntu
sudo apt-get install libmagickwand-dev
# on mac
brew install imagemagick
sudo vi /etc/ImageMagick-6/policy.xml
comment: &amp;lt;policy domain=&amp;quot;path&amp;quot; rights=&amp;quot;none&amp;quot; pattern=&amp;quot;@*&amp;quot; /&amp;gt;


# to ensure TextClip works
brew install imagemagick

# BeautifulSoup4 | bs4
pip install BeautifulSoup4
&lt;/code&gt;&lt;/pre&gt;
&lt;hr&gt;
&lt;h3 id=&#34;7-install-sublime&#34;&gt;7. Install Sublime&lt;/h3&gt;
&lt;pre&gt;&lt;code class=&#34;language-Bash&#34;&gt;# On Linux (need to pay?)
wget -qO - https://download.sublimetext.com/sublimehq-pub.gpg | sudo apt-key add -

echo &amp;quot;deb https://download.sublimetext.com/ apt/stable/&amp;quot; | sudo tee /etc/apt/sources.list.d/sublime-text.list
# OR
echo &amp;quot;deb https://download.sublimetext.com/ apt/dev/&amp;quot; | sudo tee /etc/apt/sources.list.d/sublime-text.list

sudo apt-get update
sudo apt-get install sublime-text
&lt;/code&gt;&lt;/pre&gt;
&lt;hr&gt;
&lt;h3 id=&#34;8-setup-logmein&#34;&gt;8. Setup LogmeIn&lt;/h3&gt;
&lt;pre&gt;&lt;code class=&#34;language-Bash&#34;&gt;wget http://www.vpn.net/installers/logmein-hamachi_2.1.0.165-1_amd64.deb
sudo dpkg -i logmein-hamachi_2.1.0.165-1_amd64.deb 
sudo hamachi login
sudo hamachi attach ***@gmail.com
sudo hamachi create &amp;lt;server_name&amp;gt; &amp;lt;password&amp;gt;
&lt;/code&gt;&lt;/pre&gt;
&lt;hr&gt;
&lt;h3 id=&#34;9-setup-openai&#34;&gt;9. Setup OpenAI&lt;/h3&gt;
&lt;pre&gt;&lt;code class=&#34;language-Bash&#34;&gt;conda create -n openai python=3.5
source activate openai
pip install gym
# if you want to use Breakout-ram-v0, please install atari
pip install gym[atari]

# Install tensorflow, keras, and keras-rl
pip install tensorflow
pip install keras
pip install keras-rl
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Then, you can try the following code:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&#34;language-Python&#34;&gt;import numpy as np
import gym
from keras.models import Sequential
from keras.layers import Dense, Activation, Flatten
from keras.optimizers import Adam
from rl.agents import DDPGAgent, SARSAAgent, DQNAgent
from rl.policy import BoltzmannQPolicy
from rl.memory import SequentialMemory

ENV_NAME = &#39;Breakout-ram-v0&#39; 

# Get the environment and extract the number of actions.
env = gym.make(ENV_NAME)
np.random.seed(123)
env.seed(123)
nb_actions = env.action_space.n

# Next, we build a very simple model.
model = Sequential()
model.add(Flatten(input_shape=(1,) + env.observation_space.shape))
model.add(Dense(16))
model.add(Activation(&#39;relu&#39;))
model.add(Dense(16))
model.add(Activation(&#39;relu&#39;))
model.add(Dense(16))
model.add(Activation(&#39;relu&#39;))
model.add(Dense(nb_actions))
model.add(Activation(&#39;linear&#39;))
print(model.summary())

# Finally, we configure and compile our agent. You can use every built-in Keras optimizer and even the metrics!
memory = SequentialMemory(limit=50000, window_length=1)
policy = BoltzmannQPolicy()
dqn = SARSAAgent(model=model, nb_actions=nb_actions, nb_steps_warmup=10, policy=policy)
dqn.compile(Adam(lr=1e-3), metrics=[&#39;mae&#39;])

# Okay, now it&#39;s time to learn something! We visualize the training here for show, but this slows down training quite a lot. You can always safely abort the training prematurely using Ctrl + C.
dqn.fit(env, nb_steps=50000, visualize=True, verbose=2)

# After training is done, we save the final weights.
dqn.save_weights(&#39;dqn_{}_weights.h5f&#39;.format(ENV_NAME), overwrite=True)

# Finally, evaluate our algorithm for 5 episodes.
dqn.test(env, nb_episodes=5, visualize=True)
&lt;/code&gt;&lt;/pre&gt;
&lt;hr&gt;
&lt;pre&gt;&lt;code class=&#34;language-bash&#34;&gt;pip install tensorflow keras
conda install HDF5 h5py
pip install pydot graphviz pillow quiver_engine keras-vis opencv-python moviepy BeautifulSoup4
&lt;/code&gt;&lt;/pre&gt;
&lt;hr&gt;
&lt;pre&gt;&lt;code class=&#34;language-bash&#34;&gt;pip install mxnet-cu80mkl
&lt;/code&gt;&lt;/pre&gt;
&lt;hr&gt;
&lt;h3 id=&#34;automatically-mount-hdssd&#34;&gt;Automatically mount HD/SSD&lt;/h3&gt;
&lt;ol&gt;
&lt;li&gt;
&lt;p&gt;Type the following command to get the UUID of your HD/SSD.&lt;/p&gt;
&lt;pre&gt;&lt;code class=&#34;language-bash&#34;&gt;sudo blkid
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;``&lt;/p&gt;
&lt;p&gt;The output looks similar as the follows:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&#34;language-bash&#34;&gt;/dev/sda2: LABEL=&amp;quot;Data&amp;quot; UUID=&amp;quot;1438008138******&amp;quot; TYPE=&amp;quot;ntfs&amp;quot; PARTLABEL=&amp;quot;Basic data partition&amp;quot; PARTUUID=&amp;quot;39e88529-****-****-****-************&amp;quot;
/dev/sdb2: LABEL=&amp;quot;BigData&amp;quot; UUID=&amp;quot;3868FD7668******&amp;quot; TYPE=&amp;quot;ntfs&amp;quot; PARTLABEL=&amp;quot;Basic data partition&amp;quot; PARTUUID=&amp;quot;dec90a31-****-****-****-************&amp;quot;
/dev/sdc1: UUID=&amp;quot;4a76c292-3384-4480-9672-4c8ab9******&amp;quot; TYPE=&amp;quot;swap&amp;quot; PARTUUID=&amp;quot;7bd24d17-****-****-****-************&amp;quot;
/dev/sdc2: UUID=&amp;quot;447c124c-baf5-4a24-a2c7-ac413f******&amp;quot; TYPE=&amp;quot;ext4&amp;quot; PARTUUID=&amp;quot;a96835ae-****-****-****-************&amp;quot;
/dev/sdc3: UUID=&amp;quot;88bd1683-c1fd-4097-b3c9-71998d******&amp;quot; TYPE=&amp;quot;ext4&amp;quot; PARTUUID=&amp;quot;7184ca16-****-****-****-************&amp;quot;
/dev/sda1: PARTLABEL=&amp;quot;Microsoft reserved partition&amp;quot; PARTUUID=&amp;quot;d52eb0c9-****-****-****-************&amp;quot;
/dev/sdb1: PARTLABEL=&amp;quot;Microsoft reserved partition&amp;quot; PARTUUID=&amp;quot;bc7c1633-****-****-****-************&amp;quot;
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;``&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;Add the corresponding info. to &lt;strong&gt;/etc/fstab&lt;/strong&gt;&lt;/p&gt;
&lt;pre&gt;&lt;code class=&#34;language-bash&#34;&gt;sudo vi /etc/fstab
# add the following info. to the end of the file
UUID=1438008138****** /data/Data ntfs defaults 0 2
UUID=3868FD7668****** /data/BigData ntfs defaults 0 2
&lt;/code&gt;&lt;/pre&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;p&gt;It will automatically mount these two HDs/SSDs at the next boot.&lt;/p&gt;
&lt;/li&gt;
&lt;/ol&gt;
&lt;hr&gt;
&lt;h3 id=&#34;use-remote-jupyter-on-local&#34;&gt;Use remote jupyter on local&lt;/h3&gt;
&lt;pre&gt;&lt;code class=&#34;language-Bash&#34;&gt;ssh -L 8000:localhost:8888 username@IP

jupyter notebook --no-browser --port=8889
ssh -N -L localhost:8888:localhost:8889 user@remote_host
### list ssh  
    ps aux | grep ssh
&lt;/code&gt;&lt;/pre&gt;
</description>
    </item>
    
    <item>
      <title>ReHAR: Robust and Efficient Human Activity Recognition</title>
      <link>http://xincoder.github.io/publication/2018wacv_rehar/</link>
      <pubDate>Mon, 12 Mar 2018 00:00:00 +0000</pubDate>
      <guid>http://xincoder.github.io/publication/2018wacv_rehar/</guid>
      <description>&lt;!-- &lt;div class=&#34;alert alert-note&#34;&gt;
  &lt;div&gt;
    Create your slides in Markdown - click the &lt;em&gt;Slides&lt;/em&gt; button to check out the example.
  &lt;/div&gt;
&lt;/div&gt;


Supplementary notes can be added here, including [code, math, and images](https://wowchemy.com/docs/writing-markdown-latex/). --&gt;
</description>
    </item>
    
    <item>
      <title>SBGAR: Semantics based group activity recognition</title>
      <link>http://xincoder.github.io/publication/2017iccv_sbgar/</link>
      <pubDate>Sun, 22 Oct 2017 00:00:00 +0000</pubDate>
      <guid>http://xincoder.github.io/publication/2017iccv_sbgar/</guid>
      <description>&lt;!-- &lt;div class=&#34;alert alert-note&#34;&gt;
  &lt;div&gt;
    Create your slides in Markdown - click the &lt;em&gt;Slides&lt;/em&gt; button to check out the example.
  &lt;/div&gt;
&lt;/div&gt;


Supplementary notes can be added here, including [code, math, and images](https://wowchemy.com/docs/writing-markdown-latex/). --&gt;
</description>
    </item>
    
    <item>
      <title>UAV assisted smart parking solution</title>
      <link>http://xincoder.github.io/publication/2017icuas_uav/</link>
      <pubDate>Tue, 13 Jun 2017 00:00:00 +0000</pubDate>
      <guid>http://xincoder.github.io/publication/2017icuas_uav/</guid>
      <description>&lt;!-- &lt;div class=&#34;alert alert-note&#34;&gt;
  &lt;div&gt;
    Create your slides in Markdown - click the &lt;em&gt;Slides&lt;/em&gt; button to check out the example.
  &lt;/div&gt;
&lt;/div&gt;


Supplementary notes can be added here, including [code, math, and images](https://wowchemy.com/docs/writing-markdown-latex/). --&gt;
</description>
    </item>
    
    <item>
      <title>CASHEIRS Demo [Research]</title>
      <link>http://xincoder.github.io/project/research_casheirs/</link>
      <pubDate>Mon, 01 May 2017 00:00:00 +0000</pubDate>
      <guid>http://xincoder.github.io/project/research_casheirs/</guid>
      <description>&lt;p&gt;Programming Language:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Python&lt;/li&gt;
&lt;li&gt;Java (Android Client)&lt;/li&gt;
&lt;li&gt;PHP (Web Interface)&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;This is a prototype of our CASHEIRS image retrieval system. The initial prototype system consists of a cloud server (a laptop) and a Samsung S5 phone as a data user&amp;rsquo;s mobile device. Samsung S5 has a Snapdragon 801 chip with Quad-core &lt;a href=&#34;mailto:CPU@2.5GHz&#34;&gt;CPU@2.5GHz&lt;/a&gt; and 2GB RAM. The smartphone communicates with the server via a WiFi router. In the Android client software, Caffe APIs are called to extract CNN feature from a query image. Then, this CNN feature is converted into 128bit binary code and later encrypted. To handle complex image transformations and minimize memory usage, &lt;a href=&#34;https://github.com/square/picasso&#34; target=&#34;_blank&#34; rel=&#34;noopener&#34;&gt;Picasso&lt;/a&gt;, an image downloading library for Android, is used to download query results.&lt;/p&gt;
&lt;p&gt;The following images show some query examples.&lt;/p&gt;
&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th style=&#34;text-align:center&#34;&gt;&lt;/th&gt;
&lt;th style=&#34;text-align:center&#34;&gt;&lt;/th&gt;
&lt;th style=&#34;text-align:center&#34;&gt;&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;















&lt;figure  &gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img alt=&#34;imag1&#34; srcset=&#34;
               /project/research_casheirs/1_hu2f7ad1a1cb4854741029206578e8c11a_2286289_32757f2455bacbcbb4bced13e99a2a39.webp 400w,
               /project/research_casheirs/1_hu2f7ad1a1cb4854741029206578e8c11a_2286289_bc7bd8604a2fa8729dce010195fa7c3e.webp 760w,
               /project/research_casheirs/1_hu2f7ad1a1cb4854741029206578e8c11a_2286289_1200x1200_fit_q75_h2_lanczos_3.webp 1200w&#34;
               src=&#34;http://xincoder.github.io/project/research_casheirs/1_hu2f7ad1a1cb4854741029206578e8c11a_2286289_32757f2455bacbcbb4bced13e99a2a39.webp&#34;
               width=&#34;428&#34;
               height=&#34;760&#34;
               loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;/figure&gt;
&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;















&lt;figure  &gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img alt=&#34;imag2&#34; srcset=&#34;
               /project/research_casheirs/2_hu0ff8067ffc5be317356ba4475451c042_1859040_206a849365fe58cbde19437031e88de4.webp 400w,
               /project/research_casheirs/2_hu0ff8067ffc5be317356ba4475451c042_1859040_3ac5508248f57e2121d92ba81793e183.webp 760w,
               /project/research_casheirs/2_hu0ff8067ffc5be317356ba4475451c042_1859040_1200x1200_fit_q75_h2_lanczos_3.webp 1200w&#34;
               src=&#34;http://xincoder.github.io/project/research_casheirs/2_hu0ff8067ffc5be317356ba4475451c042_1859040_206a849365fe58cbde19437031e88de4.webp&#34;
               width=&#34;428&#34;
               height=&#34;760&#34;
               loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;/figure&gt;
&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;















&lt;figure  &gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img alt=&#34;imag3&#34; srcset=&#34;
               /project/research_casheirs/3_hu47092bff32d3933d2020415a01782a02_2188498_74ca01ce86806537a41cc64ce53f821b.webp 400w,
               /project/research_casheirs/3_hu47092bff32d3933d2020415a01782a02_2188498_0477879dcb36cf6b9f94143212052907.webp 760w,
               /project/research_casheirs/3_hu47092bff32d3933d2020415a01782a02_2188498_1200x1200_fit_q75_h2_lanczos_3.webp 1200w&#34;
               src=&#34;http://xincoder.github.io/project/research_casheirs/3_hu47092bff32d3933d2020415a01782a02_2188498_74ca01ce86806537a41cc64ce53f821b.webp&#34;
               width=&#34;428&#34;
               height=&#34;760&#34;
               loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;/figure&gt;
&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td style=&#34;text-align:center&#34;&gt;















&lt;figure  &gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img alt=&#34;imag4&#34; srcset=&#34;
               /project/research_casheirs/4_hu713238c877cc8e5161aecafdf9f6710c_2145523_24ff7f1b2c198d2cb46322a3e8450b63.webp 400w,
               /project/research_casheirs/4_hu713238c877cc8e5161aecafdf9f6710c_2145523_0781f381a46c79f3e187be8f192e6f91.webp 760w,
               /project/research_casheirs/4_hu713238c877cc8e5161aecafdf9f6710c_2145523_1200x1200_fit_q75_h2_lanczos_3.webp 1200w&#34;
               src=&#34;http://xincoder.github.io/project/research_casheirs/4_hu713238c877cc8e5161aecafdf9f6710c_2145523_24ff7f1b2c198d2cb46322a3e8450b63.webp&#34;
               width=&#34;428&#34;
               height=&#34;760&#34;
               loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;/figure&gt;
&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;















&lt;figure  &gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img alt=&#34;imag5&#34; srcset=&#34;
               /project/research_casheirs/5_hu9969634cad12fe2650c5e1cd9fd5987f_2198957_1ecb3e6d00bea92aee2bfb7b3890549f.webp 400w,
               /project/research_casheirs/5_hu9969634cad12fe2650c5e1cd9fd5987f_2198957_8f5bd93e7bbc0e30f4ddb15c73cc9004.webp 760w,
               /project/research_casheirs/5_hu9969634cad12fe2650c5e1cd9fd5987f_2198957_1200x1200_fit_q75_h2_lanczos_3.webp 1200w&#34;
               src=&#34;http://xincoder.github.io/project/research_casheirs/5_hu9969634cad12fe2650c5e1cd9fd5987f_2198957_1ecb3e6d00bea92aee2bfb7b3890549f.webp&#34;
               width=&#34;428&#34;
               height=&#34;760&#34;
               loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;/figure&gt;
&lt;/td&gt;
&lt;td style=&#34;text-align:center&#34;&gt;















&lt;figure  &gt;
  &lt;div class=&#34;d-flex justify-content-center&#34;&gt;
    &lt;div class=&#34;w-100&#34; &gt;&lt;img alt=&#34;imag6&#34; srcset=&#34;
               /project/research_casheirs/6_hu36923ebd86620d6e4811dc4ef84ef836_1239768_b01d1c3daf8756d72da1b9173b043d30.webp 400w,
               /project/research_casheirs/6_hu36923ebd86620d6e4811dc4ef84ef836_1239768_7c83de8ff27e3778577bc16cfb531589.webp 760w,
               /project/research_casheirs/6_hu36923ebd86620d6e4811dc4ef84ef836_1239768_1200x1200_fit_q75_h2_lanczos_3.webp 1200w&#34;
               src=&#34;http://xincoder.github.io/project/research_casheirs/6_hu36923ebd86620d6e4811dc4ef84ef836_1239768_b01d1c3daf8756d72da1b9173b043d30.webp&#34;
               width=&#34;428&#34;
               height=&#34;760&#34;
               loading=&#34;lazy&#34; data-zoomable /&gt;&lt;/div&gt;
  &lt;/div&gt;&lt;/figure&gt;
&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;
&lt;p&gt;My responsibilities:&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;Proposed and designed an image retrieval system.&lt;/li&gt;
&lt;li&gt;Implemeted the system including an Android Client, a Python Server and a PHP interface.&lt;/li&gt;
&lt;/ol&gt;
&lt;hr&gt;
&lt;p&gt;Published paper:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Xin Li, Qinghan Xue, Mooi Choo Chuah (2017). &lt;a href=&#34;../../publication/2017infocom_casheirs/&#34;&gt;CASHEIRS: Cloud assisted scalable hierarchical encrypted based image retrieval system&lt;/a&gt;. INFOCOM 2017-IEEE Conference on Computer Communications (INFOCOM).&lt;/li&gt;
&lt;/ul&gt;
</description>
    </item>
    
    <item>
      <title>CASHEIRS: Cloud assisted scalable hierarchical encrypted based image retrieval system</title>
      <link>http://xincoder.github.io/publication/2017infocom_casheirs/</link>
      <pubDate>Mon, 01 May 2017 00:00:00 +0000</pubDate>
      <guid>http://xincoder.github.io/publication/2017infocom_casheirs/</guid>
      <description>&lt;!-- &lt;div class=&#34;alert alert-note&#34;&gt;
  &lt;div&gt;
    Create your slides in Markdown - click the &lt;em&gt;Slides&lt;/em&gt; button to check out the example.
  &lt;/div&gt;
&lt;/div&gt;


Supplementary notes can be added here, including [code, math, and images](https://wowchemy.com/docs/writing-markdown-latex/). --&gt;
</description>
    </item>
    
    <item>
      <title>Five-in-a-Row Game [Others]</title>
      <link>http://xincoder.github.io/project/others_fivestones/</link>
      <pubDate>Thu, 05 Nov 2015 00:00:00 +0000</pubDate>
      <guid>http://xincoder.github.io/project/others_fivestones/</guid>
      <description>&lt;h1 id=&#34;detection-and-description-of-visual-attributes-for-vehicles-and-pedestria&#34;&gt;&lt;strong&gt;Detection and Description of Visual Attributes for Vehicles and Pedestria&lt;/strong&gt;&lt;/h1&gt;
&lt;p&gt;Programming Language:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Python (Server)&lt;/li&gt;
&lt;li&gt;HTML5 (Interface)&lt;/li&gt;
&lt;li&gt;JaveScript&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;In this project, we designed and implemented a very simple AI for the Five-in-a-Row Game. Black plays first if white did not just win, and players alternate in placing a stone of their color on an empty intersection. The winner is the first player to get an unbroken row of five stones horizontally, vertically, or diagonally.&lt;/p&gt;
&lt;p&gt;My responsibilities:&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;Designed and implemented a JaveScript client embedded inside an HTML5 webpage that allows two players to play five-in-a-row game.&lt;/li&gt;
&lt;li&gt;Designed and implemented a server application that judges which player wins in the end.&lt;/li&gt;
&lt;li&gt;Implemented a Python AI program that can play against a single player. The AI program at least beats a player that randomly place stones on the board.&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;The following video shows a demo of our system.&lt;/p&gt;









  





&lt;video controls  &gt;
  &lt;source src=&#34;http://xincoder.github.io/project/others_fivestones/demo.mp4&#34; type=&#34;video/mp4&#34;&gt;
&lt;/video&gt;
</description>
    </item>
    
    <item>
      <title>Vehicles and Pedestrians Detection [Research]</title>
      <link>http://xincoder.github.io/project/research_vehicle_pedestrian/</link>
      <pubDate>Mon, 01 Jun 2015 00:00:00 +0000</pubDate>
      <guid>http://xincoder.github.io/project/research_vehicle_pedestrian/</guid>
      <description>&lt;h1 id=&#34;detection-and-description-of-visual-attributes-for-vehicles-and-pedestrian&#34;&gt;&lt;strong&gt;Detection and Description of Visual Attributes for Vehicles and Pedestrian&lt;/strong&gt;&lt;/h1&gt;
&lt;p&gt;Programming Language:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;C++&lt;/li&gt;
&lt;li&gt;SQL&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;In this project, we designed and implemented a system to detect and track vehicles and pedestrains from traffic surveillance videos. Once a vehicle/pedestrain is detected/tracked, some visual descriptions, e.g. color, moving direction, etc., are analyzed. All information are stored into a local SQL database for retrieval in the future.&lt;/p&gt;
&lt;p&gt;My responsibilities:&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;Proposed and implemented a detection system for vehicles.&lt;/li&gt;
&lt;li&gt;Implemeted a tracking function.&lt;/li&gt;
&lt;li&gt;Extract visual descriptions from detected vehicles.&lt;/li&gt;
&lt;li&gt;Implemented objects retrieval functions.&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;The following video shows a demo of our system.&lt;/p&gt;









  





&lt;video controls  &gt;
  &lt;source src=&#34;http://xincoder.github.io/project/research_vehicle_pedestrian/demo.mp4&#34; type=&#34;video/mp4&#34;&gt;
&lt;/video&gt;
</description>
    </item>
    
    <item>
      <title>Family Video Surveillance System [Others]</title>
      <link>http://xincoder.github.io/project/others_family_surveillance/</link>
      <pubDate>Sun, 01 Sep 2013 00:00:00 +0000</pubDate>
      <guid>http://xincoder.github.io/project/others_family_surveillance/</guid>
      <description>&lt;h1 id=&#34;detection-and-description-of-visual-attributes-for-vehicles-and-pedestria&#34;&gt;&lt;strong&gt;Detection and Description of Visual Attributes for Vehicles and Pedestria&lt;/strong&gt;&lt;/h1&gt;
&lt;p&gt;Programming Language:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;C++ (PC server: Object and Motion Detection, Alarm module)&lt;/li&gt;
&lt;li&gt;Java (Android Client)&lt;/li&gt;
&lt;li&gt;PHP (Web Interface)&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;In this project, we designed an embedded family video surveillance system. The system consists of two parts: &lt;strong&gt;(1) a PC server&lt;/strong&gt; and &lt;strong&gt;(2)an Android client&lt;/strong&gt;.&lt;/p&gt;
&lt;p&gt;The PC server can acquire, compress, and process camera frames. Once the PC server detected an abnormal condition, e.g. a moving people, it will send an alarm to the Android client.&lt;/p&gt;
&lt;p&gt;The Android client can automatically receive the alarm information and download the corresponding video by on click.&lt;/p&gt;
&lt;p&gt;My responsibilities:&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;Designed a video survellance system that consists of ARM development board (camera), a PC server, and an Android client.&lt;/li&gt;
&lt;li&gt;Implemented the main functions of the system: people detection, motion detection, alarm module.&lt;/li&gt;
&lt;li&gt;Implemented an Android app to recieve the alarm from the server and download videos.&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;The following video shows a demo of our system.&lt;/p&gt;









  





&lt;video controls  &gt;
  &lt;source src=&#34;http://xincoder.github.io/project/others_family_surveillance/demo.mp4&#34; type=&#34;video/mp4&#34;&gt;
&lt;/video&gt;
</description>
    </item>
    
    <item>
      <title>Augmented Reality Cinema [Others]</title>
      <link>http://xincoder.github.io/project/others_ar/</link>
      <pubDate>Thu, 01 Aug 2013 00:00:00 +0000</pubDate>
      <guid>http://xincoder.github.io/project/others_ar/</guid>
      <description>&lt;h1 id=&#34;augmented-reality-cinema&#34;&gt;&lt;strong&gt;Augmented Reality Cinema&lt;/strong&gt;&lt;/h1&gt;
&lt;p&gt;Programming Language:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;C# (Unity)&lt;/li&gt;
&lt;li&gt;Unity Script (Unity)&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;In this project, we designed and implemented an augmented reality system based on tablet for Autodesk Maya. The app runs on an Android tablet records the trajectory of the virtual camera. In this way, movie directors do not need to manually set the trajectory of the virtual camera in Maya.&lt;/p&gt;
&lt;p&gt;My responsibilities:&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;Developed an Android app to interactively record the trajectory of the virtual camera based on the device sensors&amp;rsquo; data.&lt;/li&gt;
&lt;li&gt;Implemented the function of replaying the trajectory based on the recorded data.&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;The following video shows a demo of our system.&lt;/p&gt;









  





&lt;video controls  &gt;
  &lt;source src=&#34;http://xincoder.github.io/project/others_ar/demo.mp4&#34; type=&#34;video/mp4&#34;&gt;
&lt;/video&gt;
</description>
    </item>
    
    <item>
      <title></title>
      <link>http://xincoder.github.io/admin/config.yml</link>
      <pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate>
      <guid>http://xincoder.github.io/admin/config.yml</guid>
      <description></description>
    </item>
    
  </channel>
</rss>