Skip to content

Reduce dp-frontend-router's HOT-PATH log.Event() memory allocations by upto 31% (and all the way to 100%) #6

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
redhug1 wants to merge 36 commits into
base: master
Choose a base branch
from
Open

Reduce dp-frontend-router's HOT-PATH log.Event() memory allocations by upto 31% (and all the way to 100%) #6

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
36 commits
Select commit Hold shift + click to select a range
8f87877
Investigate o.attach(&e) alternative for speed
redhug1 May 3, 2020
3673e06
Add new file with updated analysis: struct{} code is to be preferred
redhug1 May 14, 2020
007989d
Benchmark log memory use
redhug1 Jul 12, 2020
0f0f8dd
Allocation reduction + benchmark 3 events like dp-frontend-router
redhug1 Jul 12, 2020
d5ce839
Improve memory allocation optimize to 31% : see BenchmarkLog4()
redhug1 Jul 13, 2020
604418d
Improve memory allocation optimize to 31% : see BenchmarkLog4()
redhug1 Jul 13, 2020
49b9085
Merge remote-tracking branch 'origin/test/optimize-allocations' into …
redhug1 Jul 13, 2020
a07610b
Add feature flag "minimumAllocs". New code now saves 66%
redhug1 Jul 14, 2020
7ce4fd3
Start replacing json.NewEncoder() with unrolled code for 75% saving.
redhug1 Jul 15, 2020
ba1bff5
Stage 3: unroll expandDataToBuf(). Savings now increased to 82%
redhug1 Jul 16, 2020
406910d
Stage 4: unroll expandHTTPToBuf(). Savings now increased to 84%
redhug1 Jul 16, 2020
a261954
Make event 2 & 3 more closely resemble dp-frontend-router output
redhug1 Jul 17, 2020
49ef777
Use optimised conversion functions. Savings now up to 87.6%
redhug1 Jul 17, 2020
62b284e
Stage 5: Use optimised conversion functions. Savings now up to 87.6%
redhug1 Jul 17, 2020
d75867b
Merge remote-tracking branch 'origin/test/optimize-allocations' into …
redhug1 Jul 17, 2020
989e6ae
Stage 6: Getting to 91.6% savings and the path to 100% savings
redhug1 Jul 17, 2020
a802a9d
Stage 7: Test dp-frontend-router with New log.go => Works + its Faster
redhug1 Jul 18, 2020
8172522
Add updated file (missed from last check-in)
redhug1 Jul 18, 2020
ebff655
Add details for code changes used for Stage 7
redhug1 Jul 19, 2020
38ed7d2
Update test instructions.
redhug1 Jul 19, 2020
055c6ea
Add writing new log to fallbackDestination.
redhug1 Jul 19, 2020
0bbb71a
Stage 8: Add TestLogNew1() and fix some problems found. 87.8% saved.
redhug1 Jul 19, 2020
8c623ff
Fix bug, Fix test code. Savings now at 87.1%
redhug1 Jul 20, 2020
c44d888
Add comparing old and new events to TestLogNew1()
redhug1 Jul 20, 2020
6bb9274
Fix original TestLog() to restore a few things it changes
redhug1 Jul 20, 2020
d03349d
Unroll expandAuthToBuf(). Add Add more test code.
redhug1 Jul 21, 2020
e7470a2
Add more tests + comments.
redhug1 Jul 22, 2020
6a7a3fa
Refactor: move most new code to new files + rename
redhug1 Jul 22, 2020
9ef8a06
Refactor: move more code from log.go to unroll_event.go
redhug1 Jul 22, 2020
a0b6381
Fix broken test
redhug1 Jul 22, 2020
fa42427
Try to fix test failure
redhug1 Jul 22, 2020
cf362d0
revert to original test setup to fix github/ci
redhug1 Jul 22, 2020
d821a75
Travis problem and new test code for Event()
redhug1 Jul 22, 2020
3347f14
Stage 9: dp-frontend-router: shorten Log by 38% => Save Money
redhug1 Jul 23, 2020
85ac392
add missing file from last commit
redhug1 Jul 23, 2020
7c1d326
Stage 10: Custom Events to achieve 100% reduction in HEAP Allocations
redhug1 Aug 4, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions go.sum
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACk
golang.org/x/net v0.0.0-20190311183353-d8887717615a/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20190222072716-a9d3bda3a223/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20191026070338-33540a1f6037 h1:YyJpGZS1sBuBCzLAR1VEpK193GlqGZbnPFnPV/5Rsb4=
golang.org/x/sys v0.0.0-20191026070338-33540a1f6037/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/tools v0.0.0-20190328211700-ab21143f2384/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs=
Loading