3TC

What Is 3TC, and How Does It Relate To Blockchain

3TC isnt a blockchain in the opinion of its author, but it is in the view of some in the business. So if it’s not a blockchain then why not, and what is it. And if it is, what is similar.

Blockchain Does Not Have A Monopoly On Proof

You don’t need the universal ledger for one party to be able to digitally prove that another party said something, directly to them, in the past. You need public key 2 way authentication, and a certification authority. You use this all the time, it’s the stuff that puts the S in https://.

That of course doesn’t now mean we can make someone do what they promised to do just because we have irrefutable proof from a SHA256 digitally signed message, we need the magic of a Bitcoin with some extras to do that. No, not blockchain, a real Bitcoin, a traded crypto currency.

There’s two issues here

  1. What does the message actually mean ?, what are we promising to do, it’s just a string of 1s an 0s really.
  2. What can  blockchain possibly make anyone do, its just a computer program. All bitcoin can force you to do, the only thing it controls, is it’s crypto-currency.

If you haven’t even got a crypto currency, if you arent actually triggering value transfer because you’ve ditched Bitcoin’s inconvenient and ecologically dirty secret called Proof of Work, then what the heck do we need a blockchain for ?. All a currency-less blockchain can do is provide evidence, set values in a database that say that someone owes you. You simply dont need a blockchain to be able to achieve this.

Blockchain isn’t private

Everything you stick onto a blockchain is there, and visible to the entire network, forever. You need to assess what potential cost to privacy you may be committing to.

Traffic analysis of static wallet addresses.

It is impossible to interact with any system in an identified or non-anonymous fashion, and be anonymous. That’s an obvious fact. Blockchain’s principal and overriding goal is to provide an incorruptible, universally ordered ledger. This is achieved at a price. If you interact with Bitcoin with the same wallet address repeatedly, your transaction patterns can be quite easily detected. Permissioned blockchains may mitigate against this to some extent by not affecting the wallet balances for pure data transfer, but activity is still detectable. With potentially massive quantities of transactions on static addresses, engraved for eternity on an immutable data store, any defense against this must be able to prove that it cannot ever be broken.

Almost all examples of blockchain usage, be they crypto-currency or privately maintained (and secured!), and which require either party to repeatedly interact with the same address or just from the same node, are liable to make their relationships in the network public. Any nefarious use of Bitcoin requires your wallet address to be regularly changed, and even then it’s far from certain your interactions will never be discovered. In a legitimate business environment that simply doesn’t make sense. You will require a consistent bank or data account address.

Detecting conspiracies in a network  is clearly a highly technical issue. But it isnt simply the preserve of GCHQ and the CIA. Techniques for network analysis are in public use across the scientific community. This next point however is not up for discussion. It’s just plain fact.

Obfuscation of smart contract input data is impossible.

Everyone at Ethereum, Hyperledger & Eris is in agreement on this matter. The parametric data in your smart contract, the parameters for the rules which are embedded in your contract, have to be public if validated on a public network. There’s no clever way around that problem that is feasible that has yet been discovered.

The only way of keeping your validated private stuff private is to employ only the parties to the contract for validating a transaction. If this number is low then that will require total consensus. Everyone has to say yes, hence everyone has a power of veto. This is the declared solution to the matter from all of the above blockchain technologies, should they implement it.

We have now completely inverted the blockchain paradigm. Nobody can force anyone to do anything (even though all you can ever do anyway is force someone to spend some of your native currency). So you still have to call the lawyers to make people stick to the deal they signed, and yet we still have the potential of a thin cloud spread across the world watching our every move if we start rolling these interactions up in any way on a universal blockchain.

I haven’t moaned on yet about scalability. There are issues with Blockchain scalability, messy ones intrinsic to the architecture that nobody has a reasonable answer to yet.

There Exists A Solution

Given the inescapable issues listed above, the only possible solution is a mini blockchain, one per contract, with a simple universal consensus strategy involving only the parties on the contract. This is trivially achieved by a policy of network saturation. It is not scalable with respect to the number of parties on any one contract, it has to be say < 20. But is essentially unbounded on the total number of parties, and transactions, on this “other kind of blockchain”, because you arent actually all on one chain.

Everyone tells everyone else on the contract, and only those on the contract, about everything that happens, using little more than standard two way public key authentication. The nodes respond to each other with the current hash of the record. We are not just broadcasting a request and recording the remote acknowledgement. The remote has to reply with evidence of its current state, i.e we know that they have an identical contract in their database to the one you, and everyone else on the contract, is looking at. Any failure, whether through software error, or through malicious or faulty operation,  will be discovered immediately.

animated

We now have a very strong level of immutability already. You just cant add an invalid record to this store as it would need to be signed by all the parties and can always retrospectively be detected. We can now employ one of the core components of the blockchain architecture, the Merkle tree. This enables detection of the deletion of any item prior to the latest valid state of any of the participants. i.e we are all as good as the best secured system on the contract, we have all the correct records, no more or less, and in the right order. When you roll in the resilience of cloud based database systems, and with significant database corruption of each and all systems required to cause an undetected fraud, we now have a system which is cryptographically all but as secure as a blockchain, but doesn’t have the unfortunate side effects of telling everyone on the network who your clients are and what the relative strength of your relationship appears to be with them, or of being forced to obey a potentially faulty computer program.

Your smart contract now isn’t irrevocable. Anyone on the contract can just turn their computer off if they don’t like whats unfolding.  That’s not the sci fi stuff that is being sold I know. Private and irrevocable smart contracts are not actually deliverable, and I don’t think you really want some magic box that could, and inevitably will become a monkey’s paw.

Is 3TC  another kind of blockchain ?.  My personal view is that you’ve lost being a blockchain the minute you drop proof of work and stop being a crypto currency. The whole point of blockchain is to make you change your wallet balance. If you havent got PoW then you dont have a crypto currency. That said,  as the only solution to strict privacy is exactly what we are doing, then if you like we can call it a blockchain. But we’re missing the really novel part  of the whole blockchain architecture,  just so you know.

Let’s Start With A Contract

Rather than taking a crypto-currency and trying to work out how we bend it to our will, perhaps a better approach is to start with a contract and work out how we can smarten it up and bless it with some of the properties of the blockchain. As discussed above, we cant have irrevocable and private contracts in an honest world. What we can have is extremely high levels of immutability, or trust, between the parties, with total privacy.

Everyone on the contract has to co-operate for modifications to be accepted, but the history of agreement is extremely secure. Whether a SHA256 signature is sufficient to make a judge side with you is another issue, and as I understand it one you will lose. But making the Internet force you to do something while not telling everyone what that thing is, is neither an achievable target using existing technology nor a desirable one anyway . What is possible is a shared record that we can all trust.

What Is A Contract ?

A contract is a series of identical documents, one for each party in the contract, each document signed by all parties. Only the parties on the contract have copies. Indeed we don’t want anyone outside this group to even know of the contract’s existence.  So what we want is some kind of database replication technology that has assurance comparable with blockchain, but with total privacy. It needs to share individual records/contracts over a dynamic network specified in the contract itself. i.e we can add new members at the start and also during the life cycle of a contract, and even remove them though they will always have a history up to the point they left.

Initially we merely wish for this document to be negotiable in a mutually consistent manner and for those edits to be first replicated to each party, and then in turn approved by each party. Note there are two stages here. The first is to replicate what is actually a proposed value for a field, this is tier 1, and then the parties have to manually (or automatically as will be discussed below) agree to that value, to vote yes, this is tier 2. However, we still then need a 3rd tier, to confirm the contract as a whole. e.g. we may have all agreed to what is on the contract so far, but one or more parties still want to add more before they sign the contract as complete.

These 3 tiers of the consensus strategy are what give the technology its name.

3TC – Three Tier Consensus

  1. Transport – the replication process
  2. Negotiation – agreement of individual fields
  3. Publication – agreement on the entire contract

Transport Layer – T1

So how does the stuff get about ?. Every entity instance in our 3TC database, i.e. a row in a table in the database, is replicated individually. There’s of course only 3 things you can do, CREATE, UPDATE or DELETE.  These requests obviously have to be executed in the same order they were on the originating system , this is of course because we need any and all parent entities to exist before we insert the child. There are also issues about resolving primary keys. All tables need to have a universalId field for this. The originating entity simply broadcasts the request, including obviously the content of the record and also the hash of the entire contract at this state. Each remote party in turn re-broadcasts essentially the same request, but with the local system’s own hash. The hashes are stored in a dialogue table for each request and hence we can see if we have even hashes for a given transaction. The network is defined by the contract itself, with the ContractParty intersection table, or Crypt. So there’s some special behaviour needed for those tables. But all the others, and we are expanding the schema regularly, work the same way.  We just define the table using the Sequelize model definition files developed as part of the Gemini project. The data is then not only automatically replicated, but is also magically instantiated in the client. The only thing left is to write some template code for the display of this entity using the ViewPoint system also developed in this project to work with Marionette.

Negotiation – T2

A principal problem in any database replication system is dealing with collisions where two of the nodes both want to write a different value to the same point in the database simultaneously. There are two simple techniques to get out of this

  1. Design the database so that everyone can have their own version to write to
  2. Lock the contract and then execute the command universally

During the live running of a contract the system is generally asynchronous throughout. Remote systems do not all have to be on line for contracts to have proposals submitted. This is because all the data you are submitting is to tables that belong to both the Contract and the Party tables. In short you all have your own version of things, you cant overwrite someone else’s Proposal, or Vote on someone else’s behalf. All this is within what is called the Negotiation or T2 tier.

Publication – T3

This only gets you so far. Eventually we need to be certain that we all have the same record. That we all are agreed on what we are agreeing on and that we can publish or at least mark this point in the contract’s history as an agreed point. For this the record has to be locked, we all have to stop changing it. All systems on the contract now have to turn up for work eventually. The architecture is resilient, they don’t all have to be on line simultaneously, but we can’t go anywhere with this contract until everyone has now signed off. This rubber stamping exercise is called the Publication or T3 tier and can for any contract be either automatic based on constraints set, or require each or some of the remote parties to manually sign or agree.

We can do this locking because we have a manageable number of nodes, all of which can reasonably be expected in a business environment to have reasonably available systems. You don’t need some special internet connection or even keep your databases online 24/7, but your systems do need to be reasonably available for a contract to be confirmed. The system is intended to work with under say 20 parties per contract. If you want greater numbers then one of the parties has to work as an aggregator. Note that each of these parties is going to be a company or institution that simply cannot store its data in some communal hub. 20 of them on the same  contract is improbable. 20 of them on a secret contract is preposterous.

Contract Structure

A contract consists of a potentially hierarchical tree of fields. Apart from providing some organization to large contracts by enabling blocks to be collapsed and expanded as required, a tree of fields enables us to perform duplication actions on blocks of fields. We call these fields Elements.

Elements have a relatively complex sub schema associated with them. They have votes as mentioned above for Tier 2, they can also have  rules applied. These rule applications have modes, which in turn can be turned on and off. I have described this in previous posts as a “Generic Contract Model”. This is misleading, It’s a Generic Element Model. Contract structure is arbitrary and entirely controlled and developed by the end user community.

As we discover new distinct patterns or “features” if you like, this model can and is being expanded. But the T1 code never needs any changes as this schema is developed, we can just evolve our generic model. These new component entities may well have special properties, in particular resulting in base rules of the system being affected. These tasks and constraints are applied using the Object Relational Mapping technology discussed below.  i.e. the tables just appear at the other end. if you need extra stuff going on, you add it using hooks for this entity.

Entity Prototypes

While a contract is a list of fields which can have children or subfields, it’s perhaps better to think of it as a list of blocks or additional subforms that need completing. These subforms need defining, and then as part of the live state of the contract, replicating. The initial definition or template block for such a form is a called a prototype, and has an associated array of replicated elements (usually blocks), such as claims or endorsements on a contract. The user gets to define their own data models, their own tables and attributes basically.

What we now have is the capacity for end users to actually design their own data. Arbitrary groups of parties can develop and arbitrate not just the content but the very structure of their data. They can produce catalogues of form and contract types to be plugged in or used in full in a new contract. The ability to abstract these, to take a starting point and then add the extra stuff you need in this specific case, again from a drop down of blocks already defined, or by creating one and possibly naming it for future use, can have profound implications.

Not That Smart, But Selectively Constrained

Being told that you cant have the hippy trippy free love of the blockchain in the Insurance world, or almost any business context where we simply don’t want our association to be made public, may come as a disappointment. But I am now going to tell you that you don’t really want really smart contracts at all. You want totally transparent ones whose rules are simply constraints on what values can be and who can edit them.  We want very solid simple and transparent logic that the users can see and control. Just by restricting who can edit what, and what they can submit, we have gone a long way.

Referential fields

In order to make this fly, we need to be able to set constraints such as “Field X must be greater than field Y”, or “Y+Z”, or “max(A,B,C)”  or any standard mathematical expression. We want it so you cant even fill X in till we have a Y. And if Y changes and X now no longer works, X should be invalidated. We want fields to be able to  have more than one constraint. And a referencing syntax to navigate this tree, and a common mathematical syntax. These features alone, while transparent and easily followed, give you most of what you need in an automated contract.

Triggers and Process Definition

We can take this a step further though, and still keep it simple, courtesy of the T2&3 concepts. When field, block or contract wide consensus is reached we can invoke a trigger. This isn’t an arbitrary piece of code, it is merely the automatic invocation of either a block copy, e.g. make a new claim form, and/or the application of a mode grouping of rules. This trigger is itself just a rule, with parameters specifying the block(s) to copy and or mode(s) to apply upon everyone accepting the value.  This allows you to do things like, upon completion of some optional field, or even a specific value in a list, determine that a sub form now needs to be filled in. This allows the user to actually define a contract life cycle process.

Blob Fields

One extended data type that I am currently implementing is to support arbitrary images or other documents. The files are stored separately by each party on their own storage system, whatever that might be (i.e you can all pick your own private cloud to put the stuff on). Only the hash of the file is stored in 3TC, any upload will create a new proposal for this field, and so we automatically have a history facility.  Insertion will cause the remote nodes to request the file directly from the host  using SFTP and a proxy via the uploading node.

We will be also able to do this with XLS files, allowing embedded spreadsheets as well as documents or image files. The cool thing we can then do is extend the referencing syntax so you can say MyXLFile.C1 or better still MyXLFile.MyLabelledCell, and then set constraints on other fields using this value. So if you have an existing process whereby a reasonably defined spreadsheet gets filled in first, we can load it and even derive values from it for use within the validation and control of the contract. We can even edit it inside the page and provide a history and rollback.

Node.JS

There are some of us, quite a lot of us now, who think that the biggest thing to happen to software engineering since perhaps an HTTP client supporting in-line images was produced, is the development and astonishing rate of adoption of the server side Javascript platform developed by Google and called Node. There are a string of reasons for this success. Most profound for myself and other engineers grappling with systems that have increasingly sophisticated client side software, which is to all intents and purposes an entirely Javascript world, is the effect of using the same language for server side tasks. I cannot explain just how much difference that makes. It obviously makes life easier, but its much deeper than that. Obstacles in architecture just seem to dissolve away. Any structural deficiencies or bugs are treated equally on either side of the fence. If a problem needs fixing properly, i.e both client and server need to up their game, than lets just get on and do it. The only drawback sometimes is remembering which computer you are supposed to be thinking about.

There are other aspects about the whole nature of the language which are also part of this. Javascript, despite its unfortunate misnomer as some kind of lightweight Java, is a fundamentally different kind of language. One regular misconception is that “Javascript isnt object oriented”, or isnt OO “enough”. Object oriented programming is the business of patterns. It should really be called pattern oriented programing to explain what we are trying to do. It is a plain fact of software engineering science that there are no patterns you can give say a Java programmer that you cant give a javascript programmer. None. There is however a massive and extremely important set of patterns, basically all the ones to do with manipulating the name space, that are an anathema to Java. In short, if you want to do cool stuff, its way easier to do it in Javascript.

Tim Berners Lee no less recently defined what he calls Atwood’s Law, in honour of Jeff Atwood, co-creator of the stack-exchange family of sites. “If it can be written in Javascript, then it will, eventually, be written in Javascript”   But dont take  Jeff or Tim’s word for it, go ask the chief engineers at every single one of the top Internet brands. Facebook have gone as far as literally banning all other forms of development, at least without a seriously good excuse.

Sequelize – An Object Relational Mapper

The core of the server side architecture is the excellent Sequelize.  You may have heard of Hibernate,  a Java thing for getting you out of having to write any more SQL. Table definitions are classes and  rows in databases are instances of an object. Sequelize is another one of these ORMs. You can now write a REST interface generator that will use the set of models defined in your Sequelize instance, so all neat and tidy. Sequelize also exposes a range of hooks that you can call at different points in an entity’s history, e.g BeforeCreate, AfterUpdate etc. In effect, all that stored procedure and triggers stuff you may have done with PostgreSQL or Oracle, is now where you want it, in javascript in your application. As a result, the T1 transport layer magic of 3TC can be implemented under the hood of an interface  used directly by the REST generator. So you may just be saving an instance of say the Votes model. Its just when you insert it a load of other stuff goes on and the two special tables Request and Dialogue will have things happen in them.

3TC also implements a Sockets.IO strategy for all models and collections defined as children of a specific model. The entire exposed REST interface and the underlying database relations are transmitted via a call to the root API. In plain English, if you go to /api you will get a dump of the entire DB definition for use in the client in working out where things are, and what messages to listen to. The combination of all this results in a model being submitted on one screen on a remote node, producing a request on the remote to any clients looking at that part of the DB to fetch the now replicated value. Its a Reactive Immutable Distributed Ledger

Summary

My colleague sends me at least one “the future is blockchain” article a day. Admittedly we are now getting a weekly “blockchain might not be all what it’s cracked up to be”, and “blockchain took all my money” and “lots of money already spent on blockchain and nothing has actually turned up thats usable yet”. My favourite “blockchain will do everything” post of today included “Nuclear launch codes”. You can imagine my reaction to that one.

I am a coder. Things have to add up. I have solved your problems with what I believe, based on the interaction I have had with the Blockchain Illuminati, is logically the only way possible. But I have also done it using technology which comprises all the good stuff that has evolved over the last 5 to 10 years on how to build scalable reactive and highly complex applications. We now have the capacity to deploy here and now on use cases that make sense and can produce savings rapidly.

But what we have is so much deeper than that. Based on the  vision of a simple transparent automated record, and the capacity for end users to design and refine their own data models and in effect applications, we can revolutionize how business is not just done, but how practices are developed. There are products you never even dreamed of waiting to be deployed using this technology. Ones where defining the process, let alone gaining consensus, just wouldn’t have been feasible without 3TC.

Do You Really Want To Be Disrupted ?

Do you have to endure centralized bodies wishing to enforce and control architecture, data standards and procedure ?. Is development and deployment of such systems and standards slow to non existent ?. Do these projects require you to commit to technologies that dont even exist yet ?. Wouldn’t you rather let your own business analysts just get on with things  from the front line, and develop and forge these prototypes yourselves with partner groups that are motivated to explore these technologies .

Using a stack of existing and contemporary technology discussed above and including some of the core cryptographic components and publicly maintained code used by blockchain, we have a way of  attacking the internal and  inter-corporate IT impasses that exist in your industry now.

This has the potential to not just disrupt the entire landscape, but fundamentally change how your industry actually works.

Advertisements