This page contains all of Sam's public writings in one stream. It has been broken it up into sections for different audiences and purposes. Some sections are empty and many of my old posts and writings are yet to be moved over to this new platform.

Fact or Fiction—Chinese Medicine's San Jiao theory?

"Theory" is certainly not the right word from the perspective of mainstream medical culture. Most times it just refers to a body location. It's mostly terminology.

The Upper Jiao is essentially the Thoracic Cavity, and the Orophalanx (mouth basically). Everything above the diaphragm relating to Digestion and Circulation, and almost all of of the organs of Perspiration. It belongs alongside Thoracic Cavity in a Scientific Occidental Medical Practitioner, as an Organ Group.

The Middle Jiao is the core of digestion—Stomach, Liver particularly the closest cells to the Hepatic Portal, "Spleen" (digests red blood cells more efficently than the liver), and the Duodenum. Sparing the colourful descriptions, this is areas where food is mostly converted .

Finally, the lower burner is mostly dealing with slow extraction of less and less useful food matter, and includes the Ilium, Colon, and—along similar lines to the filtering of waste matter—the Kidneys. But also the Adrenal Complex.

Then, there's the part which is not a Theory, either—it's a more of a conjecture, scientifically speaking. It describes digestion, and some basic interactions between organs. Some of them are direct (such as the transferring of digestate from the middle to the lower jiao), and some more conceptual.

Acupuncture has been investigated in what could be called the history section of science—for instance at Cambridge in the early 20th, Joseph Needham or Gwei–Djen Lu as he became to be known left a tome of books called Science and Civilization in China which is fascinating in its own right, but a special one called Celestial Lancets for what was called "Occidental Medicine" or even "Eastern Medical Science" at the time by Western scholars. The most detailed reference translation between Eastern and Western medical concepts I am aware of is L'Acuponcture Chinoise, though I have only rarely delved into it. There's a nice vampire story in there, I would call it something of an Easter Egg for the observant (it's in a section of Chinese Medicine dealing with unusual forms of death).

But the books I read were much simpler, and written by later authors trying to leave less cryptic, though more approachable for beginner–level tertiery students. Books such as Zang Fu by Jeremy Ross and the Foundations of Chinese Medicine by Giovanni Maciocia. I have read a few of these books, though I decided that there were more pressing problems I needed to apply my life to than the relatively intellectual pursuit of Scientific development of Chinese Medical concepts. I've left my life of scientific pontification and petty miracle working that never was to join the "one—degree war".

I still find research in the field interesting, though.

I wrote up a Wikipedia page to go along with this image, and while despite my complaints about deletionism, much of the text I wrote seems to have been left intact since I wrote it to accompany the image in 2007 or whenever it was. I'll leave it for now on my own blog, I realise it is not currently citable material and missing some due diligence with citations. But the above books are a pretty useful set of general citations for this. Perhaps someone will one day verify it better.

San Jiao Theory

'''San Jiao''' is a term found in [[traditional Chinese medicine]] (TCM), as part of modelling the workings of the human body attempted by early Chinese medical writers. References to it can be found in the oldest Chinese medical texts available, including the ?Yellow Emperor's ?Huangdi Neijing.

San Jiao has been translated as "''triple heater''," ''triple warmer'' (or ''three warmers'')," and "''triple burner''," the latter of which is probably favored because of the involvement of the San Jiao in ?metabolism. The current WHO standard term is "Triple Energizer" (TE), but many authors still prefer to use San Jiao.

The San Jiao as a body model

There are three "burning spaces":

A drawing of the body, a composition of freely available body parts and a few line drawings showing which each organ group each organ belongs to, as well as the other parts of the San Jiao theory (svg source) This is a picture I drew of all of this as I was trying to wrap my own head around what it is. All the organ parts come from other free SVG files on WikiMedia. I uploaded it to WikiMedia, but unfortunately it didn't get the right MetaData to stick around; a copyright notice apparently not being enough. It probably counts as original research, anyway.

  • The Shang Jiao (upper burner) - corresponding to the thoracic cavity. This space includes Fei (lungs) and Xin (Heart), and is associated with respiration.
  • The Zhong Jiao (middle burner) - corresponding to the upper part of the dorsal cavity. This space includes Wei (Stomach), Pi (usually called "Spleen", but better called the "nutrient uptake system", eg in ?WMS terms the Hepatic Portal System, various secretory pancreatic cells, perhaps the duodenum, portions of the lymphatic system carrying ?chyle, as well as catabolic and transaminoic functions of liver cells) and is associated with digestion.
  • The Xia Jiao (lower burner) - corresponding to the lower part of the dorsal cavity. This space includes Xiao Chang (small intestine), Da Chang (large intestine), Shen (Kidneys), Pang Guang (Bladder) and is associated with elimination.

    Other ?Zang Fu theory organs were not included in the San Jiao model.

    The Hand ?Meridian (Chinese medicine) of San Jiao Shao Yang is so called because of its generalized effects across the San Jiao. San Jiao is not an ?organ (anatomy). In fact, many Zang Fu organ translations do not directly correspond with their defined western organ.

    The Shao Yang channel is the second shallowest channel in the six divisions of ?Meridian (Chinese medicine), and its hand division - San Jiao - starts at the fingernail of the ring finger, travels up the outside center of the hand and arm, encompasses the elbow, continues to the back of the Acromio-clavicular joint (part of the shoulder), meeting with the other Yang channels at the junction of the seventh cervical and first thoracic vertebrae (GV-14 "da zhui"), before travelling up the neck to behind the ear, encompassing the external ear and terminating at outer tip of the eyebrow. Aside from Wai Guan (TE-5), its points' most common clinical uses are for local problems.

    San Jiao relationships

    In TCM theory, the San Jiao is a yang organ paired with the ?Pericardium (Zang) (Xin Bao) which is the yin organ associated with it. Yang (Fu) organs are typically hollow, whereas yin (Zang) organs are more solid. The triple burner, however, is said to be primarily energetic and does not have a physical component, unlike all the other organs in TCM. In ?dissection a body, one would not be able to find a structure that could be called the San Jiao.

    The San Jiao's Hand-Foot partner is Dan (Gall Bladder).

    San Jiao function

    The San Jiao is also said to be a metabolism mechanism similar to an old-fashioned ?water wheel that is turned by incoming water and creates ?energy for accomplishing a task, such as grinding grain in the case of the water wheel, or for metabolising and ?digestion ?food in the case of the San Jiao. The San Jiao is closely associated with the ?Spleen (Zang) functions of transformation and transportation, particularly the metabolism of incoming food. The San Jiao is also closely associated with the ?Kidney (Zang)'s function in TCM. The San Jiao, however, is not limited to one metabolism function as the spleen or kidneys are, but is a general metaboliser which can be applied to a variety of metabolism needs.

    This dual usage of San Jiao to refer to a specific metabolic function and to refer to the areas of the body is a source of confusion, and care should be taken to make it explicit which is being referred to.

    San Jiao and Wu Xing

    San Jiao is related to the fire element of the ?Five elements (Chinese philosophy).

Posted early Thursday morning, January 12th, 2012

Offsetting my Air Travel (and then some)

As many people know, I tend to fly around the world a lot. A long time ago, back in about 2006 or 2007 I think, I signed up with Dopplr and put my travel in there. I did this mostly so I could hook up with friends I knew online as I travelled around the world. Well, this only actually resulted in actual meetups with friends once or twice I can remember, notably a few cocktails with [Randal Schwartz](http://www.stonehenge.com/merlyn/ in Hawai'i during our respective travels across the Pacific in opposite directions.

But one thing it did get me in the habit of doing is recording my travel accurately. Especially air travel. And at some point, it started having carbon footprint reporting functionality integrated.

It's not 100% accurate, but it's a good approximation. For instance, the precise carbon footprint of an air flight would be dependent on the model of aircraft, how full loaded the aircraft was, even the flight time, path, etc. But such precision is not necessary for its use as a crude footprint tool.

But it did let me know that since I've been recording my travel, it has been been responsible for 25 tonnes of CO₂ emissions. I think I offset just one flight before.

Well, I just went and offset all 25 Tonnes of it. Of course buying offsets is all about buying them from the right place, and I found a great directory: the Carbon Catalog, which seemed to be very inclusive and included sites I'd heard positive reviews of before. Wary of the possibility that some programmes are not providing real offsets, I looked into a few of them, and preferred:

  • projects which I understood how the offsets worked to really result in a lower CO₂ levels in the world,

  • projects which had independent certifications, in particular the certification known as the "Gold Standard"

  • projects which did not sell credits for the lowest price, under the "too good to be true" principal.

  • companies based in countries that have an emissions trading scheme

The projects I chose were priced around US$10 - $20 per tonne, so my 25 tonnes come to about $350. Not bad really, especially if you consider that the actual cost of those flights and travel over those years could have been as high as $10,000.

And in fact, because it's Christmas, I went and bought my descendents a present - 25 tonnes of CO₂ offset I didn't need to purchase.

I chose these programmes:

  • TerraPass - they have three programs, and I bought 10,000lb of offsets from each of them.

  • Carbonfund.org - they have a variety of projects, and a relatively cheap (but certainly not dodgy cheap) price per unit, and I bought 10 tonnes from them from their "basket"

  • Climate friendly - an Australian company, I bought 4 tonnes from them.

  • South Pole Carbon Asset Management Ltd - a Swiss company, with emphasis on certification. I bought a total of 24 tonnes from this company, with 4 projects - a Taiwanese wind farm, an Indonesian geothermal plant, as well as reforestation projects in Uganda and Columbia.

I haven't yet crunched the numbers to see whether or not I'm carbon neutral or not, but my rough understanding is that air travel is by far the most substantial contribution to one's carbon footprint that there is. Plus there's all my travel before 2007 - notably two and a half return trips to the UK from New Zealand I need to offset - which I'll get to in time.

In the mean time, I've offset the lion's share of my carbon footprint for myself, and for one other person in the world like me who doesn't yet believe in this stuff, or at least not enough to spend good money on it.

And my new year's resolution? Well, all my fitness stuff is already taken care of, so I can't count that twice. So, I guess my new resolution can be to keep being carbon-negative.

I invite all those who are concerned about the future of the planet to join me in this pledge of double-offsetting.

Posted in the wee hours of Sunday night, December 26th, 2011
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

By indicating your acceptance, you understand, agree, warrant and
covenant as follows:

2012 Desert International and Sprint Triathlon Accident Waiver and
Release of Liability (AWRL)


I acknowledge that this athletic event is an extreme test of a
person’s physical and mental limits and carries with it the potential
for death, serious injury and property loss.  The risks include but
are not limited to, those caused by terrain, facilities, temperature,
weather, condition of athlete, lack of hydration, equipment, vehicular
traffic, and actions of other people, including but not limited to
participants, volunteers, spectators, and event officials.  I hereby
assume all risks of participation in this event. I acknowledge that
this, Athletic Waiver and Release of Liability (AWRL) form, will be
used by the event director and event sponsors and that it will govern
my actions and responsibilities at said event(s).


I realize that liability may arise from negligence or carelessness on
the part of the persons or entities being released from dangerous or
defective equipment, property owned, maintained or controlled by them
or because of their possible liability without fault.


I certify that I am physically fit, have sufficiently trained for
participation in the event and have not been advised otherwise by
qualified medical personnel.


In consideration of my application and permitting me to participate in
this event. I hereby take action for myself, my executors,
administrators, heirs, next of kin, successors and assigns as follows:
(A) Waive, Disclaim from any and all liability for my death,
disability, personal injury, property damage, property theft and
actions of any kind which may hereafter accrue to me including my
traveling to and from this event, the following entities or persons:
Klein and Clark Sports, LLC, Greg Klein, Brenda Clark, the City of La
Quinta, Riverside County, Coachella Valley Water District, CA Hwy
Patrol, Riverside County Sheriff, all municipal agencies whose
property and/or personnel are used and other sponsoring and
co-sponsoring company(s), agency(s), or individual(s). (B) Indemnify
and Hold Harmless the entities or persons mentioned in the paragraph
from any and all liabilities or claims made as a result of
participation in this event, whether caused by the negligence of
releases or otherwise.


I hereby consent to receive medical treatment, which may be deemed
advisable in the event of injury, accident and/or illness during the
event.


The AWRL, shall be construed broadly to provide a release and wavier
to the maximum extent permissible under applicable law.  As a
participating athlete, I certify that all the information provided on
my race entry form is true and complete.  I have read this AWRL and
certify my compliance by my signature below.




ACTIVE REGISTRATION AGREEMENT AND LIABILITY WAIVER (the "Agreement and
Waiver")

1. Authority to Register and/or to Act as Agent. You represent and
warrant to The Active Network, Inc. ("Active") that you have full
legal authority to complete this event registration on behalf of
yourself and/or any party you are registering (the "Registered
Parties"), including full authority to make use of the credit or debit
card to which registration fees will be charged.  As used in this
Agreement and Waiver, Active refers to The Active Network, Inc. and
any and all subsidiaries, affiliated entities, or entities that
control or are controlled by Active singly or together and its
officers, employees, contractors, subcontractors and agents.

If you are registering a child under the age of 18 or an incapacitated
adult you represent and warrant that you are the parent or legal
guardian of that party and have the legal authority to enter into this
agreement on their behalf and by proceeding with this event
registration, you agree that the terms of this Agreement and Waiver
shall apply equally to all Registered Parties. By registering a child
under 13, you agree and consent to the collection of that child's
information which you provide for the purposes of registration.

2.  Waiver.

YOU UNDERSTAND THAT PARTICIPATION IN THE EVENT IS POTENTIALLY
HAZARDOUS, AND THAT A REGISTERED PARTY SHOULD NOT PARTICIPATE UNLESS
THEY ARE MEDICALLY ABLE AND PROPERLY TRAINED.  YOU UNDERSTAND THAT
EVENTS MAY BE HELD OVER PUBLIC ROADS AND FACILITES OPEN TO THE PUBLIC
DURING THE EVENT AND UPON WHICH HAZARDS ARE TO BE EXPECTED.
PARTICIPATION CARRIES WITH IT CERTAIN INHERENT RISKS THAT CANNOT BE
ELIMINATED COMPLETELY RANGING FROM MINOR INJURIES TO CATASTROPHIC
INJURIES INCLUDING DEATH.  YOU UNDERSTAND AND AGREE THAT IN
CONSIDERATION OF BEING PERMITTED TO PARTICIPATE IN THE EVENT, YOU AND
ANY REGISTERED PARTY, THE HEIRS, PERSONAL REPRESENTATIVES OR ASSIGNS
OF YOU OR THE REGISTERED PARTY DO HEREBY RELEASE, WAIVE, DISCHARGE AND
CONVENANT NOT TO SUE ACTIVE FOR ANY AND ALL LIABILITY FROM ANY AND ALL
CLAIMS ARISING FROM PARTICIPATION IN THE EVENT BY YOU OR ANY
REGISTERED PARTY.

3. Limitation of Liability; Disclaimer of Warranties. 

ACTIVE SHALL NOT BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
SPECIAL OR CONSEQUENTIAL DAMAGES, RESULTING FROM (A) THE USE OR THE
INABILITY TO USE ACTIVE OR (B) FOR THE COST OF PROCUREMENT OF
SUBSTITUTE GOODS AND SERVICES OR (C) RESULTING FROM ANY GOODS OR
SERVICES PURCHASED OR OBTAINED OR TRANSACTIONS ENTERED INTO THROUGH
ACTIVE OR (D) RESULTING FROM UNAUTHORIZED ACCESS TO OR ALTERATION OF
YOUR TRANSMISSIONS OR DATA, INCLUDING BUT NOT LIMITED TO, DAMAGES FOR
LOSS OF PROFITS, USE, DATA OR OTHER INTANGIBLE, EVEN IF ACTIVE HAS
BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES. YOU EXPRESSLY AGREE
THAT USE OF ACTIVE IS AT YOUR SOLE RISK. ACTIVE IS PROVIDED ON AN "AS
IS" AND "AS AVAILABLE" BASIS. ACTIVE EXPRESSLY DISCLAIMS ALL
WARRANTIES OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING WITHOUT
LIMITATION ANY WARRANTY OF MERCHANTABILITY, FITNESS FOR A PARTICULAR
PURPOSE OR NON-INFRINGEMENT.

Active makes no warranty that the Active sites' services will be
uninterrupted, secure or error free. Active does not guarantee the
accuracy or completeness of any information in, or provided in
connection with, the Active sites. Active is not responsible for any
errors or omissions, or for the results obtained from the use of such
information. You understand and agree that any material and/or data
downloaded or otherwise obtained through the use of the Active sites
is at your own discretion and risk and that you will be solely
responsible for any damage to your own computer system or loss of data
that results from the download of such material and/or data.

4. Indemnification. You agree to indemnify and hold each of Active
harmless from and against any and all damages, costs, claims or
demands, including reasonable attorneys' fees, made by any third party
due to or arising from or relating to your use of Active or the
violation of any term of this Agreement and Waiver as well as the
Terms of Service located at:
http://www.activenetwork.com/terms-of-use.htm by you.

5. Applicable Law; Consent to Jurisdiction. The Active sites
(excluding linked sites) are controlled by Active from its offices
within the State of California, United States of America. By
completing this event registration, both you and Active agree that the
statutes and laws of the State of California, without regard to the
conflict of laws principles thereof, will apply to all matters
relating to this event registration, this Agreement and Waiver, the
Terms of Use or other use of the Active sites. You agree that
exclusive jurisdiction for any dispute with Active resides in the
courts of the State of California and you further agree and expressly
consent to the exercise of personal jurisdiction in the courts of the
State of California in connection with any dispute including any claim
involving Active or its affiliates, subsidiaries, employees,
contractors, officers, directors, telecommunication providers and
content providers.

6. Severability. You further expressly agree that this Agreement and
Waiver is intended to be as broad and inclusive as is permitted by the
law of the State of California and that if any provision of this
Agreement and Waiver shall be found to be unlawful, void, or for any
reason unenforceable, then that provision shall be deemed severable
from this Agreement and Waiver and shall not affect the validity and
enforceability of any remaining provisions.

BY INDICATING YOUR ACCEPTANCE OF THIS AGREEEMENT AND WAIVER, YOU ARE
AFFIRMING THAT YOU HAVE READ AND UNDERSTAND THIS AGREEMENT AND WAIVER
AND FULLY UNDERSTAND ITS TERMS.  YOU UNDERSTAND THAT YOU ARE GIVING UP
SUBSTANTIAL RIGHTS, INCLUDING THE RIGHT TO SUE.  YOU ACKNOWLEDGE THAT
YOU ARE SIGNING THE AGREEMENT AND WAIVER FREELY AND VOLUNTARILY, AND
INTEND BY YOUR ACCEPTANCE TO BE A COMPLETE AND UNCONDITIONAL RELEASE
OF ALL LIABILITY TO THE GREATEST EXTENT ALLOWED BY LAW.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (Darwin)

iQEcBAEBCgAGBQJO4nVEAAoJEBdtaL3wGtIoPAsH/2OSeg1Zni7xu1TRGq+Yx4wO
JsLHbugLdcuHtDFOq8cBuTXYCqLjKyLsH2YzD9IcQAf4pbbdHLt0/6SwmyjevvZs
aliruP0yZxDLzJXJJRhgdrV4R0V0XVzXsZY/IAiesIQrE76TLNR7yFjw6lcFdbXT
Xl5/Lrnxai6m8o4TPROSxyChfFfu8XS5Jl6nVkU4EKGL7eczok3Pu3oh1p8u5eIi
2TH68iEIHM10HO0Pr6jEGLru42TjqHkGe8/evQzLv82n4GJ2N1qv+MOgmWVUCLZJ
uT+h9MQF9C89J8e2EH8dX7u5fcT8+dWV/6SladtSEUa+mupYKR6mNAgK7JaJW7Q=
=pyxs
-----END PGP SIGNATURE-----
Posted late Friday evening, December 9th, 2011

La Quinta 2012 Signature

Verification instructions for Unix command-line:

$ gpg --recv-key 0x176D68BDF01AD228
gpg: requesting key F01AD228 from hkp server pgp.net.nz
gpg: key F01AD228: "Sam Vilain <sam@vilain.net>" not changed
gpg: Total number processed: 1
gpg:              unchanged: 1
$ curl http://vilain.net/talk/fit/la_quinta_2012/waiver.html |
     gpg --verify
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 15203  100 15203    0     0  26596      0 --:--:-- --:--:-- --:--:-- 35438
gpg: Signature made Fri Dec  9 12:53:24 2011 PST using RSA key ID F01AD228
gpg: Good signature from "Sam Vilain <sam@vilain.net>"
$
Posted late Friday evening, December 9th, 2011

SamV back-catalog now available

Just realised that a bunch of my projects are not online any more. Well, now they are back up via git dumb http at http://git.utsl.gen.nz/ - gitweb to follow.

Posted Wednesday evening, June 15th, 2011

Was that a "hit" ?

The big earthquake just been. Did Ken Ring's predictions "hit" it?

According to my previous betting schedule, I wagered these bets:

  • 3 chips on ±1 day surrounding the lunar equinox [ paying 14:1 ]
  • 3 chips on ±1 day surrounding the lunar perigee [ paying 14:1 ]
  • 2 chips on ±1 day surrounding the point of maximum declination [ paying 14:1 ]
  • 2 chips on ±1 day surrounding the full moon [ paying 14:1 ]
  • 2 chips on ±1 day surrounding the new moon [ paying 14:1 ]
  • 1 chip on ±1 day surrounding the point of first quarter [ paying 14:1 ]
  • 1 chip on ±1 day surrounding the point of third quarter [ paying 14:1 ]
  • 2 chips on ±1 day surrounding the lunar apogee [ paying 14:1 ]

Let's just assume for now that the chance of the earthquake is completely random (it isn't; there is a 2% extra chance at lunar perigee, but let's not worry about this for now).

The earthquake was (in GMT/UTC) at 13 June 2011 at 02:20.

The lunar perigee (closest point to earth in its orbit) was at Jun 12, 2011 01:43:00 AM GMT - miss by all of 37 minutes

The point of southern lunar declination was at Jun 15, 2011 8:54am GMT - miss by 1d 6h 34m.

The full moon was at June 15, 20:14 GMT (very shortly, along with the eclipse of course) - miss by 1d 18h 34m.

Darn. Didn't win any pots.

Let's widen the odds to ±2 days:

  • 3 chips on ±2 days surrounding the lunar equinox [ paying 7:1 ]
  • 3 chips on ±2 days surrounding the lunar perigee [ paying 7:1 ]
  • 2 chips on ±2 days surrounding the point of maximum declination [ paying 7:1 ]
  • 2 chips on ±2 days surrounding the full moon [ paying 7:1 ]
  • 2 chips on ±2 day surrounding the new moon [ paying 7:1 ]
  • 1 chip on ±2 days surrounding the point of first quarter [ paying 7:1 ]
  • 1 chip on ±2 days surrounding the point of third quarter [ paying 7:1 ]
  • 2 chips on ±2 days surrounding the lunar apogee [ paying 7:1 ]

The paid odds reduce, because the windows are wider.

The 3 chips on the perigee paid off, returning 21 chips. All other bets failed, returning nothing. 16 chips were laid down. Total profit: 5 chips.

What about widening the odds to ±3 days, making the odds 3.5:1?

Then, the 3 bet on the perigee, the 3 on the full moon, and the 2 bet on the declination all pay back. Again 16 chips were laid down. This pays back 28 chips, a more substantial profit, in fact almost doubling the outlay for this month.

I guess when gambling, sometimes you win. To enjoy it the most, you need to forget all the times that you went home empty-handed instead.

Posted Wednesday evening, June 15th, 2011 Tags:

Wild-Type Outer Membrane Vesicle Vaccines

Vaccines. They're putting hobbled viruses and bacteria into our system so that they can cause an infection, and thereby build an immunity, right? Like Smallpox and Cowpox, right?

Well, sometimes. That is called a Live Attenuated vaccine. They're difficult to manage and relatively high risk.

There are many different types of vaccine, all of which involve hobbling the rogue organism to different degrees.

If they get cooked, nuked (that is, sterilized in an irradiation facility such as the one in Upper Hutt), or perhaps killed with strong salt, acid, base, or some such - then it's called an inactivated vaccine. These are a step up.

They still work, even though the vaccine or bacterium does not cause an infection. They do not replicate in the body. Any reaction that occurs, will be an immune response only. All that means is that your immune system is responding to the foreign matter, like it is when you get a pimple or your hayfever plays up.

A diagram showing the formation of an Outer Membrane VesicleA vesicle is a little bubble of cellular membrane. It has all of the external markers or antigens, without the dangerous bacterium inner.

A step up again from an inactivated vaccine is a sub unit vaccine. If you, through chemical manipulation, extract just a part of the organism - then you have the safest known approach to creating vaccines today. They are lower in side effects, easier to manage (as there is no live agent to keep alive), although as a real infection does not occur, several shots are needed to provide immunity. If there was a "real" infection, it would provide the body with a steady stream of greeblies to fight.

The type of vaccine used for Mengingococcal disease (group B, the type causing NZ's epidemic since 1991) is called wild-type outer membrane vesicle or wtOMV in the medical literature.

It is something of a testament to the success of modern medical science that society demands almost perfect results from it. But these are relatively early days for the wtOMV approach. It is extremely specific and targeted to a particular strain of a disease. It may not provide indefinite protection; but through testing it can be shown that it can provide protection for long enough to stop an outbreak.

Scientists are working on this. I read with interest the abstract of a paper on the performance of vaccines including MeNZB:

There is good reason to believe that in the coming few years the “OMV-concept” will be exploited further and that a number of cross-protective “universal” antigens will be included in vaccines against serogroup B meningococcal disease.

This is extremely promising. We're not there yet, and so when vaccines didn't work 100%, it's a good idea not to start hating on the scientists or the institutions supporting them. I mean, certainly make them work for their money and stuff but don't forget that most of these scientists are simply motivated individuals who want to use their talent and skills to make a difference. Look at the far-flung institutes that produced that paper: Norwegian Institute of Public Health, Oslo, Norway; Institute of Environmental Science and Research, Porirua, New Zealand; Victoria University, Wellington, New Zealand; Finlay Institute, Havana, Cuba; Novartis Vaccines & Diagnostics, Siena, Italy; Ministry of Health, Wellington, New Zealand. That in itself speaks volumes about the independence of their work.

Why is the MeNZB Immunity only Temporary?

A chart showing the Humoral Immune Response chartHumoral Immune Response is shown here; other forms of defence include innate immunity (a wide category including everything from enzymes in tears to the skin) and phagocytosis.

The full picture of immunity is quite a complex picture; but nonetheless it is well known; one of the things that was certainly very striking to me as I was studying First Year Anatomy & Physiology was the level of detail that science knows. I was quite surprised to know how much information was known, even in the decade-old book I was using as a text.

But basically, immunity is only permanent if it goes through a particular stage of the immune response. I won't pretend to remember the intricate details, but in essence you don't get the strongest protection without a real infection, and real infections present the greatest danger - perhaps death with some diseases.

What about averse reactions?

You might already be primed to attack that material being injected to you. You might already have acquired immunity, in which case the reaction will be much more immediate - your system knows these foreign particles and the alarm bells go off. This will cause a reaction.

This is especially the case with N. meningitidis. It is considered normal flora in your throat - 5-15% of people are expected to have it. The throat has an organ called the tonsils which allow the body to prepare itself for infection by fighting the bacteria in the food you eat. So, it's quite expected that a good number of people will already be immune and therefore elicit a stronger immune response.

But it's temporary, and links between vaccine responses and autism have been not only debunked but revealed as fraudulent. Don't let yourself be duped!

What about breast feeding?

For the first 6-12 months of your life, you have antibodies (one mechanism of immune response) floating in your blood which were transferred from the mother. This is called innate immunity, but these antibodies are not transferred via breast-milk; the baby's digestive system will destroy them before they can!

It's all a vast Big Pharma conspiracy!

Cartoon: Opponents of the Cercical Cancer Vaccine speak out

For a start, pharmaceutical companies are probably actually undermining their medium-term takings by producing vaccines. Wouldn't it be cheaper for them not to produce them at all, and instead sell us remedies when we fall sick?

I have very little time for complaints like this in New Zealand, where there is Pharmac for bulk purchasing power. I just don't believe that the staff at Pharmac - some of whom I have known and talked to in the past - can be duped by sales execs or that the independent monitoring panels would fail to pick anything up.

That said, I certainly disagree with direct to consumer marketing, and from a macro-economic perspective, the pressure from maintaining high profit margins by our current implementation of capitalism can appear (and often even be) sinister.

The fight against these diseases is the raw fight for existence, which in modern times is already stepping outside of our frames of reference. We need concerted action on this, not half-hearted. It's a war-scale effort - don't defect to the enemy!

Posted Thursday night, April 14th, 2011 Tags:

Vaccines and MeNZB

There's a big anti-vaccination movement around at the moment, and I think it's ill-advised and extremely unhelpful to society.

I recently had a friend die of Meningococcal disease. Not a very close friend, to a person I know she was like a second daughter, well by analogy she was to me like a second cousin I saw at larger family gatherings.

However as she died at a ridiculously young age and was otherwise in good health, I found it somewhat disturbing and so started to research on this phantom menace disease. I knew little about it other than it must be one of those classes of disease which infect the dorsal cavity (the surrounding of which is called the meninges) which contain the spinal cord and brain.

A scanning electron microscope picture of a Neisseria meningitidis bacteriumA scanning-electron microscope (I assume) picture of N. meningitidis

Turns out, that there was a New Zealand epidemic of this disease caused by a bacterium ( Neisseria meningitidis) which, according to the Wikipedia article on it, is found in 5-15% of adults. Quite how it goes from being apparently harmless flora in the throat to infecting the blood and meninges is still a mystery.

So this means that the disease can be controlled in two, perhaps three ways: firstly by controlling the spread of the flora by getting people to be more aware of things like sharing saliva and handwashing, to reduce the chances that you have any of the disease in your throat. Secondly, you can immunise, so that if the bacterium does get into your bloodstream, your immune system has a better chance to repel it in time. Lastly, anything to boost the general health of the immune system might help.

A scanning electron microscope picture of a Neisseria meningitidis bacteriumA picture from Nature showing N. meningitidis is gram negative

There are many groups of the disease, A, B, C, W135, X ... most of these have simple, cheap polysaccharide vaccines and a new conjugate vaccine for type A has recently been developed. These are relatively low-cost and are used in African countries.

However the New Zealand outbreak was primarily group B. So, this requires a more technical counter where you basically have to strip the bacteria of their outer membrane vesicles, mix those outer bits with some adjuvant and a buffer solution, then dilute the whole lot in saline. When injected, it will usually stimulate the immune system into fighting an easy fight against the tiny pieces of the bacteria - an in doing so, tool them up to be able to fight the real thing should it ever have to.

A labeled diagram of the meningococcal cell wallStructure of the meningococcal cell wall. Capsular polysaccharides and outer membrane proteins are the target of ongoing vaccine research. (from chori.org)

It's not 100%. Perhaps for practical reasons they had to select a single dominant strain. It can go wrong in a number of places. The immune system might not respond to the vaccine and so immunity might not be conferred. This is a common misunderstanding, people have this assumption that once you get the shot you are immunised for life and if you catch the disease then the immunisation must be a fraud.

Fighting epidemics is fighting statistics. It's frustrating because it's very hard to see if it worked, and if you are trying multiple approaches, which one was more effective. To get the most benefit, when there is an epidemic, you target the vulnerable group - in this case, people who are likely going to be swapping saliva with the most number of different partners intentionally or inadvertently - in this case, under 30's.

A diagram of the meningococcal NZ disease outbreak vs vaccinated percentageThis graph comes from an anti-vaxxer, but note the sharp drop in incidence as the vaccination is more widely taken up.

That doesn't mean that such statements imply that people who die of this dreadful disease were swapping saliva more than any other person. It doesn't mean that when I call for wider immunisation that I am casting aspersions on the parents of my friend. She was immunised. Everyone says shining things about her.

What I'm calling for is co-operation with the medical field and people to stop passing around these poorly founded arguments against vaccinations, for it helps no-one to simply give up on one of our key approaches to combating epidemics.

In particular, if you are the sort of parent who chooses not to immunize, because of fears about a minor rash or because it's not 100%, you are detracting from the value of the immunization programme. You are detracting far more than the share of the population represented, you include a percentage of the people they come into contact with. That's the network effect in reverse. It gets shittier the more people who opt out.

I'm angry at the 17% of parents who said NO to the programme. I'm angry at the one third of NZ health professionals who think there is a link between Autism and MMR vaccine but are too lazy to research for the benefit of spreading better information. I'm angry at Sue BradfordKedgeley for supporting the anti-vaccination cause.

I guess my main point is that the argument against vaccinations that says that we shouldn't immunise, because you know children need a few good diseases to harden themselves up, pretty much falls flat when those diseases cause sudden death. Worrying about vaccine side-effects or big pharma conspiracies is a First-World Problem, and acting on them has a cost in blood.


Posted Monday night, April 11th, 2011 Tags:

Ravings to Hypothesis: a stab at a "Ring function"

As others have pointed out, Ken Ring makes a lot of predictions. As Ring does not attempt to turn his viewpoints into rigourous studies, the best that work like this normally deserves is referencing the closest rebuttal of similar ideas along with some reasoning as to why it is sufficiently similar. Not every crackpot idea deserves a full scientific investigation! But the supermoon hypothesis is now gaining momentum, at least in the media.

Ring does make some quantitative predictions, let's look at perhaps this section from a book of Ring's, Moon & Weather Lore:

Earthquakes mostly occur when the perigeal or apogeal moon is at either declination(stitial colure) or crossing the equator (lunar equinox), and within one or two days of either of these. A detailed glance at any earthquake gathering station will reveal that around these dates the numbers of quakes rise steeply and then dropoff afterwards as the moon moes out of those declination zones. Close perigee and full Moon or new moon adds to the potential for increased earthquake activity along the moon's path between the latitudes.

And later, from Predicting the Weather by the Moon:

There is evidence that moonquakes increase when the Moon is closest in its orbit to the Earth. Correspondingly, we might expect an increase in Earthquakes at that time, (the perigee) too. Earthquakes are triggered by the moon in its monthly movement north and south of the quator and its orbit around the Earth. The word ‘triggered’ is used here because the Moon may pass over a danger point many times until the stress on a fault becomes too great, after which the fault may give in one sudden movement.

...

One of the main danger times is when the Moon is crossing the equator during the monthly declination cycle. This is the time while the Moon is moving quickly between the hemispheres. When the Moon is at the maximum 28° declination, it will cross the equator twice each month at about seven degrees in a day which gives considerable pull on the planet. At minimum 18°, it crosses at about four degrees in a day and the effect is less positive.

The other danger point is while the Moon is at either of the maximum declination positions north and south of the equator. The Moon is at these positions for about three days and does place considerable strain on the techtonic plates while there. It must be remembered that the Moon is always on the move and a quake can happen at any time.

...

In G.A. Elby's book “Earthquakes” (Heinemann 1980), 209 earthquakes dating back to 1505 were recorded with their dates. We can check each quake against Moon phases. 96%. of these quakes recorded which were above 6 on the Richter Scale, occurred exactly on or within a day of one extreme feature of the Moon cycle, that is, New Moon, Full Moon, Apogee or Perigee. 75% involved two factos; when the say, the Perigee plus Full or New moon were on the same day.

So what's the summary of this?

Ring actually predicts Earthquake risk as a function over time and space. Each of the major events in the cycle of the moon's orbit represent increased risk factors; a confluence of extremes, further risk again.

This might not exactly correlate with past studies into lunar earthquake relationships; a study which fails to find a pattern only disproves its own method - it does not exclude other studies which might use a different method.

That being said, it's not a good idea to ignore studies which produce negative results; they shave away at the idea, approaching it from many different angles, until the balance of probabilities is that the idea is considered disproven.

Testing Ring's theory scientifically.

Perhaps the simplest approach to testing a prediction theory would be a gambling game, similar to roullette.

The goal of this is to show, scientifically, that a certain limited sets of dates have an increased risk of earthquake. If a significant relationship is found, these dates could potentially be used as quasi-arbitrary dates for civil defence planning.

Ladies and Gentlemen, bets please.

As the model makes predictions, the ‘house’ pays odds on those predictions based on the odds of that prediction occurring according to best accepted theories.

You are allowed to place this pot on future times, and possibly, specific regions (which would lengthen the odds dramatically).

Where the earthquake events occur, they pay back to the bet placed on them, times the released energy in units of, say, log E, where E is the energy released by the earthquake in MegaJoules, such that getting a hit on a very large earthquake pays back handsomely.

Odds would shorten after an earthquake for aftershocks in the region, according to current established theory. For some types of analysis, it may be more useful to simply remove aftershocks from the input data to avoid having to make prediction functions include aftershock predictions.

The nice thing about this is it allows people to "play" real-time, as well as being able to test the past success of the forecasting abilities (a technique known as hindcasting). Thus people who are not able or willing to share their methods can participate and be judged fairly.

Gambler's ruin

A function which simply places bets on all outcomes is not useful and will result in a high score. There are several ways around this:

  1. the betting agent has a finite pool of chips which they can only bet until they have run out; the house scores in terms of paid back chips, and functions are compared by the amount of chips they have in their pool. The nice thing about this is that it is somewhat self-regulating. However it is more difficult to reason with.

  2. placed bets are scaled so that they represent a predicted energy release function, with the total energy release matching the energy release of the period. The "overlap" is the score of the function.

  3. A statistical significance test. Feed the prediction function or predictions with random input data (with known aftershock behaviour built into it) and see how many times it gets such a high score. If the score run against the real data is never matched by the prediction function with random input data, over say 100 runs, then this may be enough to show statistical significance at the 1% level.

A concrete "Ring Function";

The function will bet on a series of extremes and mid-points:

  • 3 chips on ±1 day surrounding the lunar equinox
  • 3 chips on ±1 day surrounding the lunar perigee
  • 2 chips on ±1 day surrounding the point of maximum declination
  • 2 chips on ±1 day surrounding the full moon
  • 2 chips on ±1 day surrounding the new moon
  • 1 chip on ±1 day surrounding the point of first quarter
  • 1 chip on ±1 day surrounding the point of third quarter
  • 2 chips on ±1 day surrounding the lunar apogee
  • 3 extra chips on the lunar perigee bets covering the 2 closest lunar approaches every 18.6 years
  • 3 extra chips on the lunar apogee bets covering the 2 furthest lunar distances every 18.6 years
  • 3 extra chips on the bets covering the maximum declination on years where the declination is at its maximum 28° (as in 2004/5)
  • 3 extra chips on the best covering the lunar equinox, on the bets that surround the maximum declination times (ie, the points where the change in declination peaks).

The above should be able to test the claim from Predicting the Weather by the moon - though obviously, there is still a lot of mathematical work involved in getting the rules of the game ironed out, and expressing the functions mathematically.

To give the man a chance, the values of the odds, and the width of the bets, are to be fine-tuned using a genetic algorithm; ie, tweak values one by one to get better results.

Anyone want to write the site/thesis?

The Christchurch Daily Energy Release page has a chart which is something towards the above goals.

However a full system to incorporate existing theories, a betting/odds engine (the minimal approach above), as well as collecting/calculating all the relevant information and writing the functions in terms of those sources is just not a small job. Ken Ring, of course, could fund this, if he was genuine about testing his theory and not just after selling predictions. Then again, it's perhaps not in his best interests financially to do this - as he writes in his book above, he's already confirmed to his own satisfaction that there is a pattern. So he'll keep his "competitive edge" rather than try to solve the massive problem of proving a hunch scientifically.

I produce this text in the hope that those who do try to take up the task of proving or disproving Ring's predictions have something to work with, without being forced to purchase or read his books, which are full of crazy theories. And I'm more than happy to send my copies of the books to someone seriously taking this challenge on.

But myself, I have theories to develop in my own field.

Posted Friday night, March 11th, 2011 Tags:

The GitTorrent Commit Reel

The commit reel is defined in section 5 of the GitTorrent RFC.

It is defined as an uncompressed stream of objects, sorted in a particular way. In practice, it is only the commit objects that are sorted, and all of the dependent objects for those commits are placed with the commit which first introduces them.

So, you start with a repository:

a horizontal chart of a project history

You sort the objects so that they are in reverse date order (tie breaking is still required over git rev-list --date-order, as well as fetching their types and sizes, to produce the commit reel index.

SHA1 hashtypesizeinfo
e951c3b45579blob971lib/VCS/Git/Torrent/Tracker.pm
4a39b387218etree38lib/VCS/Git/Torrent
46a6dd40761eblob1797lib/VCS/Git/Torrent.pm
cb169dea8427tree72lib/VCS/Git
6856da5de8a8tree30lib/VCS
e028c2ec652ftree30lib
a8c6175cb855tree30
6d669a0d7649commit177
d7934d77db6dblob508lib/VCS/Git/Torrent/PWP/Message.pm
831a2dce3123tree38lib/VCS/Git/Torrent/PWP
b67f62af3325blob2062lib/VCS/Git/Torrent/PWP.pm
8e49bb567004tree102lib/VCS/Git/Torrent
d9cfbd2965e1tree72lib/VCS/Git
760c03b92584tree30lib/VCS
58e8231290fatree30lib
08d6743bc1cdtree30
6e85df39b2e9commit233
ae59d4c6cdadblob239t/91-pod-coverage.t
...
9f21fdc6b232commit504
7ed81b753c34blob528lib/VCS/Git/Torrent/Reference.pm
111a3c708d42tree321lib/VCS/Git/Torrent
32f0b74a2902blob6311lib/VCS/Git/Torrent.pm
da591fe54883tree72lib/VCS/Git
7b702d0cf7detree30lib/VCS
39ec1765b517tree30lib
6e5bb34706f6tree245
5e8f6a7807a3commit277

a commit reel

Then, you take the total size of the "tape" and divide by the number of blocks you require. Let's go with 4 for this example.

a horizontal chart of a project history, broken into 4 segments

The listing from the test commit in VCS::Git::Torrent has a total of 233141 bytes of uncompressed object data. Let's divide that into 4 segments on 58285 byte boundaries:

Chunk 1 Chunk 2 Chunk 3 Chunk 4
6d669a0d7649 commit 3145
6e85df39b2e9 commit 6250
d16fe9b37f1c commit 7269
b9b5df08c542 commit 10216
9f5380b003fc commit 13715
3d954bf97808 commit 15211
53b2a50ab357 commit 64934
f8a02453062d commit 76844
60f7c92ec68f commit 78718
8e4c833bc0ed commit 90027
9595e4d0ed4a commit 99113
2499769d4e5b commit 113780
2b67a6d1898a commit 116380
c24dcdcd46de commit 158557
bffe789b4a13 commit 162339
cc77ed21cf03 commit 164454
1dfd53badd66 commit 170494
497da251f9dc commit 174642
5b7e980dce4b commit 178961
6c1fd6467f49 commit 183229
ae4aee0f484e commit 187522
69ff2248cf7f commit 191852
40149c3f6e62 commit 199468
93083bfcc5ee commit 202889
4ff65c62c570 commit 209765
76ed2bbc552c commit 214713
9f21fdc6b232 commit 225327
5e8f6a7807a3 commit 233141

The testpacking.pl script in the VCS::Git::GitTorrent distribution can generate these lists and show how much bandwidth is wasted by using 4 separate packs:

$ git update-ref refs/heads/oldeg 5e8f6a7807a3
$ perl bin/testpacking.pl -n4 oldeg
Generating index...
Length is 233141, 4 blocks of 58286 each
do_pack(3d954bf97808)
Slice #0 (up to 58286): 15211 => 6554 (43%)
do_pack(2b67a6d1898a 9595e4d0ed4a --not 3d954bf97808)
Slice #1 (up to 116572): 101169 => 30035 (29%)
do_pack(497da251f9dc --not 2b67a6d1898a 9595e4d0ed4a)
Slice #2 (up to 174858): 58262 => 16951 (29%)
do_pack(5e8f6a7807a3 --not 497da251f9dc 9595e4d0ed4a)
Slice #3: 58499 => 10224 (17%)
Overall: 233141 => 63764 (27%)
vs Bundle: 233141 => 58297 (25%)
Overall inefficiency: 9%
$ 

So what this is saying is that our repository, originally a 58k bundle, can be split into 4 chunks, defined by the listed boundary commits. At the end, you get 4 bundles of varying sizes, with an extra 5k, or 9% of overhead (yes, these packs are thin).

So that's the idea anyway. To run the above example, you can clone the github repository, and install the requisite modules via CPAN:

$ git clone git://github.com/samv/vcs-git-torrent.git \
        VCS-Git-Torrent
...

$ cd VCS-Git-Torrent

$ perl Makefile.PL ...

$ make ... $

If it complains about missing modules, install via CPAN:

$ cpan Test::Depends Bencode IO::Plumbing
...
Update: Figures for my git.git clone:
arcturus:~/src/git$ time perl ../VCS-Git-Torrent/bin/testpacking.pl -n32 master maint pu
missing fields on Reference at /usr/lib/perl5/Class/MOP/Mixin/AttributeCore.pm line 53
Generating index...
Length is 1104821033, 32 blocks of 34525658 each
do_pack(7e011c40bc6c 466fede1bdfd 76a8323ac7f5)
Slice #0 (up to 34525658): 34518888 => 1909503 (5%)
do_pack(cf1fe88ce1fb b3f041fb0f7d a9572072f0ab fdeb2fb61669 --not 7e011c40bc6c 466fede1bdfd 76a8323ac7f5)
Slice #1 (up to 69051316): 34529558 => 1417850 (4%)
do_pack(38035cf4a51c 1b83ace35e78 50b44eceed21 2326acfa95ac --not cf1fe88ce1fb b3f041fb0f7d a9572072f0ab fdeb2fb61669)
Slice #2 (up to 103576974): 34528221 => 1243468 (3%)
do_pack(c7162c1db6fe b642d9ef6433 ada5853c98c5 --not 38035cf4a51c 1b83ace35e78 f2f880f53707 50b44eceed21 2326acfa95ac cf1fe88ce1fb)
Slice #3 (up to 138102632): 34483917 => 1109044 (3%)
do_pack(f16db173a468 f25b79397c97 61ffbcb98804 8c6ab35efe63 3d234d0afacd efffea033457 53cda8d97e6e da7bad50ed08 c27d205aaefb 96bc4de85cf8 8e27364128b0 a0764cb838c2 b1e9fff7e76c 5faf64cd28bf --not c7162c1db6fe b642d9ef6433 2e1ded44f709 ada5853c98c5 cf1fe88ce1fb)
Slice #4 (up to 172628290): 34429886 => 898299 (2%)
do_pack(a2540023dcf8 3159c8dc2da4 5a03e7f25334 ab41dfbfd4f3 e4fe4b8ef7cd 9c7b0b3fc46e a06f678eb998 d0b353b1a7a2 d0c25035df48 18b0fc1ce1ef 1729fa9878ed 1f24c58724a6 f2b579256475 937a515a15f7 --not f16db173a468 f25b79397c97 61ffbcb98804 8c6ab35efe63 3d234d0afacd efffea033457 53cda8d97e6e da7bad50ed08 c27d205aaefb 96bc4de85cf8 8e27364128b0 a0764cb838c2 b1e9fff7e76c 5faf64cd28bf cf1fe88ce1fb)
...
do_pack(607a9e8aaa9b e39e0d375d1d 106a36509dc7 0e098b6d79fb 14c674e9dc52 43485d3d16e4 7a4ee28f4127 118d938812f3 cc580af88507 86386829d425 3b5ef0e216d2 36e4986f26d1 41fe87fa49cb 9e4b7ab65256 3deffc52d88d b53bb301f578 ad17f01399a9 17635fc90067 375881fa6a43 --not f8b5a8e13cb4 50ff23667020 345a38039414 3f721d1d6d6e 977e289e0d73 2ff4d1ab9ef6 69932bc6117d 1d7b1af42028 fcdd0e92d9d4 754ae192a439 3eb969973335 0cd29a037183 6e0800ef2575 df533f34a318 32d86ca53195 f0cea83f6316 4e65b538acc9 3cb1f9c98203 0eaadfe625fd cf1fe88ce1fb)
Slice #29 (up to 1035769740): 35405510 => 677049 (1%)
do_pack(609621a4ad81 eab58f1e8e5e e7e55483439b 46e09f310567 134748353b2a 500348aa6859 a4ca1465ec8a d23749fe36f1 c8998b4823cb 4d23660e79db ad3f9a71a820 b1a01e1c0762 24ab81ae4d12 c591d5f311e0 9f67d2e8279e 2aae905f23f7 a75d7b54097e 86140d56c150 9bccfcdbff3b 02edd56b84f0 204d363f5a05 7c85d2742978 a5ca8367c223 46148dd7ea41 b7b10385a84c a099469bbcf2 fe0a3cb23c79 6b87ce231d14 1ba447b8dc2e 9fa708dab1cc 1414e5788b85 aa43561ac0c1 63267de2acc1 --not 607a9e8aaa9b e39e0d375d1d 106a36509dc7 30ae47b4cc19 e9c5dcd1313d 51ea55190b6e d5f6a96fa479 0e098b6d79fb 14c674e9dc52 43485d3d16e4 7a4ee28f4127 118d938812f3 cc580af88507 86386829d425 3b5ef0e216d2 36e4986f26d1 41fe87fa49cb 9e4b7ab65256 3deffc52d88d b53bb301f578 ad17f01399a9 17635fc90067 375881fa6a43 3cb1f9c98203 cf1fe88ce1fb)
Slice #30 (up to 1070295398): 34563903 => 641336 (1%)
do_pack(8644f69753e0 --not 609621a4ad81 eab58f1e8e5e e7e55483439b d52dc4b10b2f ebc9d420566d f740cc25298e 492cf3f72f9d 46e09f310567 134748353b2a 500348aa6859 a4ca1465ec8a d23749fe36f1 c8998b4823cb 4d23660e79db ad3f9a71a820 b1a01e1c0762 24ab81ae4d12 c591d5f311e0 9f67d2e8279e 2aae905f23f7 a75d7b54097e 86140d56c150 9bccfcdbff3b 02edd56b84f0 204d363f5a05 7c85d2742978 a5ca8367c223 46148dd7ea41 b7b10385a84c a099469bbcf2 fe0a3cb23c79 6b87ce231d14 1ba447b8dc2e 9fa708dab1cc 1414e5788b85 aa43561ac0c1 63267de2acc1 17635fc90067 cf1fe88ce1fb)
Slice #31: 34528236 => 603576 (1%)
Overall: 1104821033 => 26021211 (2%)
vs Bundle: 1104821033 => 23888867 (2%)
Overall inefficiency: 8%

real	16m54.074s
user	4m30.961s
sys	11m9.302s

That's dividing the pack defined by three branches into 32 generally evenly-sized chunks. Actually the chunks at the beginning are larger than the later ones, which are all between 500kB and 950kB. While they are not perfectly sized, at least they can be generated by any node with the underlying objects, without transferring a binary pack.

However, what will matter is that execution time; the Perl prototype is needlessly inefficient. With a revision cache, we should be able to reduce that time drastically and hopefully be able to retrieve the boundary commits for a given range of commits and number of chunks in milliseconds; the remaining work is mostly on git pack-objects, but given we've drastically reduced the work it has to do, the overall load on the network should not be drastically higher; and because peers can potentially trade these blocks, the workload can be spread out.

Posted in the wee hours of Wednesday night, March 10th, 2011