tinyxxx/index.html at master · tinygrad/tinyxxx · GitHub

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
<!DOCTYPE html>
<html lang="en">
<head>
  <meta charset="UTF-8">
  <title>tinygrad: A simple and powerful neural network framework</title>
  <style>
    body {
      font-family:'Lucida Console', monospace;
      font-size: 18px;
      color: #111;
      background-color: white;
      max-width: 1280px;
      margin: 0 auto;
      padding: 0 50px;
      box-sizing: border-box;
    }
    .landing {
      display: flex;
      flex-direction: column;
      align-items: center;
      justify-content: center;
      min-height:100vh; min-height:100dvh;
      text-align: center;
    }
    svg { width: 100%; height: auto; }
    .svg-container { width: 100%; max-width: 400px; display: inline-block; }
    .separator { letter-spacing: -9px; }
    hr { border: none; height: 2px; background-color: lightgrey; width: 100%; margin: 30px auto; }
    table { border-collapse: collapse; text-align: center; width: 100%; }
    td {
      padding: 5px 20px;
      border: 1px solid #dddddd;
    }
    .quiet, .links-container a { color: inherit; }
    .links-container { font-size: 24px; margin: 20px; }
    .links-container a { text-decoration: none; margin: 0 10px; }
    .product-photo { display: block; max-height: 640px; margin: 0 auto; }
    .faqtable dt { margin-bottom: 0.75em; font-style: italic; }
    .faqtable dd { margin-left: 0; margin-bottom: 2em; }
  </style>
</head>
<body>
  <div class="landing">
    <div class="svg-container">
      <svg viewBox="0 0 130 50" xmlns="http://www.w3.org/2000/svg">
        <!-- t -->
        <rect x="10" y="0" width="10" height="40" fill="#000000" />
        <rect x="0" y="10" width="30" height="10" fill="#000000" />
        <rect x="10" y="30" width="20" height="10" fill="#000000" />
        <!-- i -->
        <rect x="40" y="0" width="10" height="10" fill="#000000" />
        <rect x="40" y="20" width="10" height="20" fill="#000000" />
        <!-- n -->
        <rect x="60" y="10" width="10" height="30" fill="#000000" />
        <rect x="60" y="10" width="20" height="10" fill="#000000" />
        <rect x="80" y="20" width="10" height="20" fill="#000000" />
        <!-- y -->
        <rect x="100" y="10" width="10" height="20" fill="#000000" />
        <rect x="100" y="20" width="30" height="10" fill="#000000" />
        <rect x="120" y="10" width="10" height="30" fill="#000000" />
        <rect x="100" y="40" width="20" height="10" fill="#000000" />
      </svg>
    </div>
    <div class="links-container">
      <a href="#tinygrad">tinygrad</a><span class="separator">|</span>
      <a href="https://docs.tinygrad.org">docs</a><span class="separator">|</span>
      <a href="#worktiny">jobs</a><span class="separator">|</span>
      <a href="#tinybox">tinybox <span style="color:orange">(buy now!)</span></a><span class="separator">|</span>
      <a href="#faq">FAQ</a>
    </div>
  </div>

  <hr>
  <h2 id="tinygrad">tinygrad</h2>
  <p>We write and maintain <a href="https://github.com/tinygrad/tinygrad">tinygrad</a>, the fastest growing neural
    network framework</p>

  <p>It's extremely simple, and breaks down the most <a
      href="https://github.com/tinygrad/tinygrad/blob/master/examples/llama.py">complex</a> <a
      href="https://github.com/geohot/tinygrad/blob/master/examples/stable_diffusion.py">networks</a> into 3 <a
      href="https://github.com/geohot/tinygrad/blob/master/tinygrad/uop/ops.py">OpTypes</a></p>

  <b>ElementwiseOps</b> are UnaryOps, BinaryOps, and TernaryOps.<br/>
    They operate on 1-3 tensors and run elementwise.<br/>
    example: SQRT, LOG2, ADD, MUL, WHERE, etc...<br/><br/>
  <b>ReduceOps</b> operate on one tensor and return a smaller tensor.<br/>
    example: SUM, MAX<br/><br/>
  <b>MovementOps</b> are virtual ops that operate on one tensor and move the data around copy-free.<br/>
    example: RESHAPE, PERMUTE, EXPAND, etc...<br/><br/>

  <p>But how...where are your CONVs and MATMULs? Read the code to solve this mystery.</p>

  <hr>
  <h2 id="worktiny">Work at tiny corp</h2>
  We <a href="https://geohot.github.io/blog/jekyll/update/2023/05/24/the-tiny-corp-raised-5M.html">are now funded</a> and <b>hiring</b> full time software engineers. Very talented interns okay.<br/><br/>
  See <a href="https://docs.google.com/spreadsheets/d/1WKHbT-7KOgjEawq5h5Ic1qUWzpfAzuD_J06N1JwOCGs/edit?usp=sharing">our bounty page</a> to judge if you might be a good fit. Bounties pay you while judging that fit.<br/><br/>
  We are also hiring for operations and hardware, but if you haven't contributed to tinygrad your application won't be considered.

  <hr>
  <h2 id="tinybox">tinybox <span style="color:orange">(now shipping)</span></h2>
  <img class="product-photo" src="assets/tinybox.jpg">
  <p>We sell a computer called the tinybox. It comes in red, green, and soon, exa.</p>
  <table>
    <tr><td></td></td><td><b style="color:red"><a class="quiet" href="https://x.com/__tinygrad__/status/1989093779073290735">red v2</a></b></td><td><b style="color:green"><a class="quiet" href="https://x.com/__tinygrad__/status/1996375079568163122">green v2 blackwell</a></b></td><td><b>exabox</b></td></tr>
    <tr><td>FP16 (FP32 acc) FLOPS</td></td><td>778 TFLOPS</td></td><td>3086 TFLOPS</td><td>~1 EXAFLOP</td></tr>
    <tr><td>GPU Model</td><td>4x 9070XT</td><td>4x RTX PRO 6000 Blackwell</td><td>720x RDNA5 AT0 XL</td></tr>
    <tr><td>GPU RAM</td><td>64 GB</td><td>384 GB</td><td>25,920 GB</td></tr>
    <tr><td>GPU RAM bandwidth</td><td>2560 GB/s</td><td>7168 GB/s</td><td>1244 TB/s</td></tr>
    <tr><td>GPU link bandwidth</td><td>full fabric PCIe 4.0 x16</td><td colspan=1>full fabric PCIe 5.0 x16</td><td>full fabric 400 GbE</td></tr>
    <tr><td>CPU</td><td>32 core AMD EPYC</td><td>32 core AMD GENOA</td><td>120x 32 core AMD GENOA</td></tr>
    <tr><td>System RAM</td><td>128 GB</td><td>192 GB</td><td>23,040 GB</td></tr>
    <tr><td>System RAM bandwidth</td><td>204.8 GB/s</td><td>460.8 GB/s</td><td>55.2 TB/s</td></tr>
    <tr><td>Disk size</td><td>2 TB fast NVMe</td><td colspan="1">4 TB raid + 1 TB boot</td><td>480 TB raid</td></tr>
    <tr><td>Disk read bandwidth</td><td>7.3 GB/s</td><td>59.3 GB/s</td><td>7.1 TB/s</td></tr>
    <tr><td>Networking</td><td>2x 1GbE + OCP3.0</td><td>2x 10GbE + OCP3.0 PCIe5</td><td>3.2 TB/s scale out</td></tr>
    <tr><td>Noise</td><td colspan="2">&lt; 50 dB, 31 low speed fans</td><td>65 db @ 10 meters</td></tr>
    <tr><td>Power Supply</td><td>one 1600W, 100V~240V</td><td colspan="1">2x 1600W, 100V~240V</td><td>600 kW, 200V~240V</td></tr>
    <tr><td>BMC</td><td>AST2500</td><td colspan="1">AST2600</td><td>custom</td></tr>
    <tr><td>Operating System</td><td colspan="3">Ubuntu 24.04</td></tr>
    <tr><td>Dimensions</td><td colspan="2">12U, 16.25" deep, 60-90 lbs</td><td>20x8x8.5 ft, 20,000 lbs</td></tr>
    <tr><td>Rack?</td><td colspan="2">Freestanding or rack <a class="quiet" href="https://rackmountmart.store.turbify.net/26slidrailfo.html">mount</a></td><td>concrete slab</td></tr>
    <tr><td>Driver Quality</td><td>Good</td><td colspan="1">Great</td><td>functions as single GPU</td></tr>
    <tr><td>SHIPPING</td><td><a href="https://tinycorp.myshopify.com/products/tinybox-red-v2">IN STOCK - $12,000</td><td><a href="https://tinycorp.myshopify.com/products/tinybox-green-v2-with-4x-rtx-pro-6000-blackwell">MADE TO ORDER - $65,000</td><td><i><a href="https://tinycorp.myshopify.com/products/exabox-preorder">coming 2027</a> - ~$10M</i></td></tr>
  </table>
  <br/>
  <center>
  for updates on products and inventory, <a href="https://tinycorp.myshopify.com/pages/mailing-list">sign up for the mailing list</a>
  </center>
  <hr>
  <h2 id="faq">FAQ</h2>
  <dl class="faqtable">
    <dt>What is a tinybox?</dt>
    <dd>It is a very powerful computer for deep learning, and likely the best performance/$. It was <a href="https://public.tableau.com/views/MLCommons-Training_16993769118290/MLCommons-Training">benchmarked</a> in MLPerf Training 4.0 vs computers that cost 10x as much. And of course, anything that can train can do inference.</dd>

    <dt>How do I get a tinybox?</dt>
    <dd>Place an order through the links above. The factory is up and running, and it will ship within one week of us receiving the payment unless otherwise specified. Currently offering pickup in San Diego + shipping worldwide.</dd>

    <dt>Where can I learn more about the tinybox?</dt>
    <dd>We have a lot of content on our <a href="https://x.com/__tinygrad__">Twitter</a>, we also have a <a href="https://docs.tinygrad.org/tinybox/">tinybox docs page</a> and a #tinybox discord channel.</dd>

    <dt>Can I customize my tinybox?</dt>
    <dd>In order to keep prices low and quality high, we don't offer any customization to the box or ordering process. Of course, after you buy the tinybox, it's yours and you are welcome to do whatever you want with it!</dd>

    <dt>Can you fill out this supplier onboarding form?</dt>
    <dd>In order to keep prices low and quality high, we don't offer any customization to the box or ordering process. If you aren't capable of ordering through the website, I'm sorry but we won't be able to help.</dd>

    <dt>Can I pay with something besides wire transfer?</dt>
    <dd>In order to keep prices low and quality high, we don't offer any customization to the box or ordering process. Wire transfer is the only accepted form of payment.</dd>

    <dt>Can I contact someone at tiny to discuss my needs?</dt>
    <dd>In order to keep prices low and quality high, we don't employ any sales people. We sell boxes through the website. The upside of this is that our pricing is simple and transparent.</dd>

    <dt>Can I get a W-9 for this purchase?</dt>
    <dd>Yes, you can <a href="assets/fw9.pdf">download that here</a>.</dd>

    <dt>Is tinygrad used anywhere?</dt>
    <dd>tinygrad is used in <a href="https://github.com/commaai/openpilot">openpilot</a> to run the driving model on the Snapdragon 845 GPU. It replaces <a href="https://developer.qualcomm.com/sites/default/files/docs/snpe/overview.html">SNPE</a>, is faster, supports loading onnx files, supports training, and allows for attention (SNPE only allows fixed weights).</dd>

    <dt>Is tinygrad inference only?</dt>
    <dd>No! It supports full forward and backward passes with autodiff. <a href="https://github.com/tinygrad/tinygrad/blob/master/tinygrad/gradient.py">This</a> is implemented at a level of abstraction higher than the accelerator specific code, so a tinygrad port gets you this for free.</dd>

    <dt>How can I use tinygrad for my next ML project?</dt>
    <dd>Follow the installation instructions on <a href="https://github.com/tinygrad/tinygrad">the tinygrad repo</a>. It has a similar API to PyTorch, yet simpler and more refined. Less stable though while tinygrad is in alpha, so be warned, though it's been fairly stable for a while.</dd>

    <dt>When will tinygrad leave alpha?</dt>
    <dd>When we can reproduce a common set of papers on 1 NVIDIA GPU 2x faster than PyTorch. We also want the speed to be good on the M1. ETA, Q2 next year.</dd>

    <dt>How is tinygrad faster than PyTorch?</dt>
    <dd>For most use cases it isn't yet, but it will be. It has three advantages:
      <li>It compiles a custom kernel for every operation, allowing extreme shape specialization.</li>
      <li>All tensors are lazy, so it can aggressively fuse operations.</li>
      <li>The backend is 10x+ simpler, meaning optimizing one kernel makes everything fast.</li>
    </dd>

    <dt>Where is tinygrad development happening?</dt>
    <dd>On GitHub and <a href="https://discord.com/invite/ZjZadyC7PK">on Discord</a></dd>

    <dt>How can the tiny corp work for me?</dt>
    <dd>Email me, george@tinygrad.org. We are looking for contracts and sponsorships to improve various aspects of
      tinygrad.</a></dd>

    <dt>How can I work for the tiny corp?</dt>
    <dd>See <b>hiring</b> above. Contributions to <a href="https://github.com/tinygrad/tinygrad">tinygrad</a> on GitHub
      always
      welcome, and a good way to get hired.</dd>

    <dt>Can I invest in the tiny corp?</dt>
    <dd>Invest with your PRs.</dd>

    <dt>What's the goal of the tiny corp?</dt>
    <dd>To accelerate. We will commoditize the petaflop and enable AI for everyone.</dd>
  </dl>
</body>
</html>