Skip to content

Commit 9327dd4

Browse files
committed
chore: update README.md
1 parent 41fdb10 commit 9327dd4

2 files changed

Lines changed: 54 additions & 10 deletions

File tree

README.md

Lines changed: 53 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -1,9 +1,19 @@
11
# C++ Refresh
22

3-
Brush up on C++ personally to be a real-time engineer in AI era. For fundamental jargon you must know, please visit [jargon.md](./docs/jargon.md).
3+
I'm currently brushing up on my C++ skills to prepare for a career as a real-time engineer in the age of AI. Check out [jargon.md] for the fundamental terms you'll need to know.
4+
5+
## Contents
6+
7+
- [Battlefield](#battlefield)
8+
- [Topics](#topics)
9+
- [Allocators & Cache Behavior (Day 05)](#allocators--cache-behavior-day-05)
10+
- [Practical Application](#practical-application)
411

512
## Battlefield
613

14+
<details>
15+
<summary> Click to expand/collapse </summary>
16+
717
```shell
818
┌──────────────────────────────┐
919
│ CPU │
@@ -26,13 +36,13 @@ Brush up on C++ personally to be a real-time engineer in AI era. For fundamental
2636
│ │ (Shared, MBs) │ │ Bigger and slower than L2
2737
│ └────────────────────────┘ │
2838
└──────────────┬───────────────┘
29-
30-
Local Memory Controller
31-
39+
40+
Local Memory Controller
41+
3242
┌──────────────────┴──────────────────┐
3343
│ │
34-
RAM (NUMA Node 0) RAM (NUMA Node 1)
35-
~80ns latency ~150ns latency
44+
RAM (NUMA Node 0) RAM (NUMA Node 1)
45+
~80ns latency ~150ns latency
3646
```
3747

3848
How latency can grow...
@@ -47,15 +57,20 @@ Instruction →
4757
miss → RAM (NUMA remote)
4858
```
4959
60+
</details>
61+
5062
## Topics
5163
52-
[day01](./src/cpp/day01/): stack vs heap
53-
[day02](./src/cpp/day02/): reference vs copy
54-
[day03](./src/cpp/day03/): elide vs move vs copy
55-
[day04](./src/cpp/day04/): STL Containers & API Design
64+
- [day01](./src/cpp/day01/): stack vs heap
65+
- [day02](./src/cpp/day02/): reference vs copy
66+
- [day03](./src/cpp/day03/): elide vs move vs copy
67+
- [day04](./src/cpp/day04/): STL Containers & API Design
5668
5769
## Allocators & Cache Behavior (Day 05)
5870
71+
<details>
72+
<summary> Click to expand/collapse </summary>
73+
5974
### What an allocator actually is
6075
6176
An `allocator` answers two questons:
@@ -113,3 +128,31 @@ struct Good {
113128
```
114129
115130
Group hot data together.
131+
132+
</details>
133+
134+
## Practical Application
135+
136+
Based on my experience, fine-tuning pre-trained AI models for specific applications is becoming increasingly straightforward, thanks to the optimization of inference frameworks, particularly on GPUs.
137+
138+
As model inference becomes faster and more efficient, the true bottleneck often shifts to data flow and real-time decision-making within the Python-based container. Python's inherent inefficiencies in the post-processing layer, especially on hot paths, can significantly hinder performance.
139+
140+
In AI-heavy applications—such as video streaming, autonomous driving, smart cities, and trading—optimizing everything beyond inference is critical. C++ plays a key role in eliminating these inefficiencies and squeezing out those final milliseconds. Therefore, I will simulate the Python hot path for processing object detection metadata and re-implement it in C++ to achieve the performance gains needed for real-time applications.
141+
142+
The directory [./src/python/yolo/inference](./src/python/yolo/inference/) will simulate Ultralytics YOLO's inference and generate dummy metadata, which is used in the analytics layer (In real-world applications, this metadata loop is handled by NVIDIA DeepStream, which is highly optimized).
143+
144+
The Python implementation will serve as a reference for the analytics pipeline, which I will later re-implement in C++ to optimize the performance of time-critical operations in real-time systems.
145+
146+
```shell
147+
# Inference container is just to simulate metadata generation in quicik.
148+
# In real world applications, I used the highly optimized NVIDIA DeepStream.
149+
+----------------------------+ +----------------------+
150+
| Inference Container | | Analytics Container | : This is where I will re-write
151+
| (Dummy Ultralytics YOLO) | | (Processing Metadata)|
152+
| | | |
153+
| - Pretend YOLO Inference | | - Process Metadata |
154+
| - Generate Dummy Metadata | ----> | - Analytics Logic |
155+
| | | |
156+
+----------------------------+ +----------------------+
157+
```
158+

src/python/_cli.py

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -69,4 +69,5 @@ def main(
6969

7070
return
7171

72+
# Run the default simulation
7273
run_simulation(config_data["yolo"]["fps"])

0 commit comments

Comments
 (0)