The future of Dapps programming with lightwallet & INFURA

I have been so busy lately ! 

I finally find some time to write an article that, I hope, will help a lot of you guys creating great DApps. 

How to use Web3js with INFURA 

So this is a new trend that I personally find pretty cool. The idea is to do 100% of the identity / transaction management in the browser and sending the signed transactions directly. 

Although in theory this approach is pretty awesome, it is still very cutting edge and the tools are not mature yet to work with them without any issues. 

Disclaimer

This is my experience with doing this. I'm not pretending my approach is the best and actually I would love to hear your feedback. How can I improve that? 

Dependencies

eth-lightwallet: This is the library I have used to generate the accounts. Right now it was used to generate accounts by using a user generated seed but hopefully in the future it will be able to use other means.
ethereumjs-tx: This library is needed to stringify the transaction so you can sign it.  
web3: This is the web3js library
hooked-web3-provider: This library is used to add code so you can sign the transactions locally instead of the Ethereum node
web3-provider-engine: This is also needed to tweak web3js so that most of the handling is done locally.

Configuration

const Transaction = require('ethereumjs-tx');
const ProviderEngine = require("web3-provider-engine");
const Web3Subprovider = require("web3-provider-engine/subproviders/web3.js");
const Web3 = require("web3");
const CacheSubprovider = require('web3-provider-engine/subproviders/cache.js');
const FixtureSubprovider = require('web3-provider-engine/subproviders/fixture.js');
const FilterSubprovider = require('web3-provider-engine/subproviders/filters.js');
const VmSubprovider = require('web3-provider-engine/subproviders/vm.js');
const HookedWeb3Provider = require("hooked-web3-provider");
const NonceSubprovider = require('web3-provider-engine/subproviders/nonce-tracker.js');
const RpcSubprovider = require('web3-provider-engine/subproviders/rpc.js');

Here are all the dependencies you require to use it all. 

Now it is time to put all of it together:


var seed = // put here your selected seed
    const accountPwd = 'this is my password'; //you can set a password here
    lightwallet.keystore.deriveKeyFromPassword(accountPwd, function (err, pwDerivedKey) {
        if (err) {
            console.error(err);
        }
        var ks = new lightwallet.keystore(seed, pwDerivedKey);
        // generate five new address/private key pairs
        // the corresponding private keys are also encrypted
        ks.generateNewAddress(pwDerivedKey, 1);
        self.accounts = ks.getAddresses();
        self.ks = ks;
        self.pwDerivedKey = pwDerivedKey;
        web3.eth.defaultAccount = add0x(self.accounts[0]);
    });
//lightwallet result can be saved as a JSON in the local storage or anywhere else. I haven't done that for my tests
// configure web3js
function add0x (input) {
      if (typeof(input) !== 'string') {
        return input;
      }
      else if (input.length < 2 || input.slice(0,2) !== '0x') {
        return '0x' + input;
      }
      else {
        return input;
      }
}

engine.addProvider(new FixtureSubprovider({
        web3_clientVersion: 'ProviderEngine/v0.0.0/javascript',
        net_listening: true,
        eth_hashrate: '0x00',
        eth_mining: false,
        eth_syncing: true
    }));

    // cache layer
    engine.addProvider(new CacheSubprovider());

    // filters
    engine.addProvider(new FilterSubprovider());

    // pending nonce
    engine.addProvider(new NonceSubprovider());

    // vm
    engine.addProvider(new VmSubprovider());
    var web3;
    // id mgmt
    const provider = new HookedWeb3Provider({
        host: nodeUrl, //this would be your INFURA url Morden or Main
        transaction_signer: {
            // Can be any object that implements the following methods:
            hasAddress: function (address, callback) {
                callback(null, true); //This means that every address is accepted. You could put some security here if needed
            },
            signTransaction: function (tx_params, callback) {
                var txOptions = {
                    gasPrice: web3.eth.gasPrice.toNumber(),
                  //For test purposes, this is the limit I have put. You could evaluate the call and set the estimated value here.
                    gasLimit: 3000000,
                    to: add0x(tx_params.to),
                    nonce: add0x(tx_params.nonce),
                    data: add0x(tx_params.data),
                    value: add0x(tx_params.value)
                };
                var rawTx = (new Transaction(txOptions)).serialize().toString('hex');
                var signedTx = lightwallet.signing.signTx(self.ks, self.pwDerivedKey, add0x(rawTx), self.web3.eth.defaultAccount);
                callback(null, add0x(signedTx));
            }
        }
    });

    web3 = new Web3(provider);

    // data source
    engine.addProvider(new RpcSubprovider({
        rpcUrl: nodeUrl
    }));

    // log new blocks
    engine.on('block', function (block) {
        console.log('================================')
        console.log('BLOCK CHANGED:', '#' + block.number.toString('hex'), '0x' + block.hash.toString('hex'))
        console.log('================================')        
    });

    // network connectivity error
    engine.on('error', function (err) {
        // report connectivity errors
        console.error(err.stack)
    });

// start polling for blocks
engine.start();

Important points

As you can see, I'm using a function called add0x. I had issues because sometimes, some values (such as addresses) where missing the "0x" in the beginning. The issue is that the error message is very misleading. So beware of where "0x" is needed and where it isn't. 

P.S: There are differences between Parity and Geth on this subject

Let's use web3js

And now you can use web3js :) 

First you need to create an interface for you smart contract

//the ABI has to be an object, not a string
const MyContract = web3.eth.contract(abi);
const MyContractInstance = MyContract.at(MyContractAddress);

Now you can use web3js as if you were using a "normal" node with RPC. 

Important Point

Again there is something to be aware of here, with this hooked web3js, you cannot do blocking calls. If you try, you will get an exception telling you that it is not possible.

Now what I didn't know is what does it mean exactly. It simply means that you HAVE TO give a callback function as the last parameter of your function call.

//here is an example of an integration with Angular and how I translate callback functions into Promises
function callMyConstantFunction() {
 var defer = $q.defer(); //This could be translated into a more standard Promise creation
     MyContractInstance.callMyConstantFunction.call(web3.eth.defaultAccount, function (err, result) {
            if(result.toNumber() === 0) {
                defer.resolve(null);
            }else {
                defer.resolve(result.toNumber());    
            }
        });
return defer.promise; 
}

Another point I tend to forget, "uint" returns a special object called BigNumber and you have to call toNumber() to get the actual value. 

Et voilà

And now you can have fun with your smart contracts and INFURA without worrying about your local node and more importantly, being able to create mobile web applications !!

Credits

I would like to thank Dan Finlay who helped me tremendously understanding how it works. I wouldn't have made it without him and his experience. 

Keep calm and code on

This post is a therapeutic one. I need to vent, I need to let it out.

CAN WE PLEASE STOP WITH THIS "ETHEREUM IS GOING TO DIE" RHETORIC ??

Ok deep breathe ... Breathe in ... Breathe out ...

So I need to let a few things out. My first and most frustrating one is, can we please have some kind of proper standard for discussion and fact checking? I mean I get that when it comes to value and money. people get emotional but this is ridiculous. 

I thought we were a group of educated science loving people, but this looks more and more like a Republican convention !

Here are some of the common pattern that I can't stand anymore!

I told you so

Ok every guy on the internet, I am sorry to contradict you, but the fact that one event that you predicted happened does NOT mean that you are a prophet nor that you understand what the hell is going on. Will Ethereum be here in 6 months? Will ETC be here? Nobody knows. And I know you want to look cool and show how knowledgeable you are but usually you are not. All this is new, it is the first time this happens and unless you come with very hard evidence that your model or theory is correct, you are not better than any low level politician. 

Ethereum is doomed because xxx

Really? The project that will change the world is finished because it has done something you don't like? ok ... Or maybe this is more wishful thinking than proper analysis. Taking 2-3 "facts" and extrapolate without being too rigorous just make you look either dumb or with an agenda.

The "Bailout"

Can we please stop using words with strong emotional  connotations regardless to their actual meaning? I am ready to discuss the implications of the Hard Fork with anybody, I even think that we need this constructive discussion to move forward and make sure the project will be more and more awesome everyday. But if you say bailout, you lost me. You either don't know what it means or you don't care ... either way this is dumb

Discussion standard

So first of all, let me say that I am not naive and I know very well that there is no way that people will follow these discussion standards. People will always stay the same and they won't change because I write it in a post. But in these times where people drive me crazy by rebashing the same stories over and over again, I have faith that some people will connect with my struggle and I can maybe meet some people with a positive and civil attitude. 

Definition

So before everything, we should try to define what we mean when we say something subjective or non precise. For example if we say that we predict that the value will surge, what do you mean exactly? from 10 to 100$? from 10 to 20$? from 10 to 11$ ? A lot of discussions where around the definition of "immutability" and "decentralization" but very often those terms are coming as self explanatory. I know I am right because everybody has the same understanding than me of this word. Guess what? This is not very constructive ! It is impossible to get anywhere if we don't make sure we are talking about the same thing.

Let's be civil / solution oriented

I get it. A lot of money is at stake. We are here to change the world, not just fool around.  But we don't care if you are right or not? This is not the point! You are insignificant! Please repeat after me, I am insignificant. 

Your opinion does not matter. All that matters is if the project is a success or a failure. All that matters is whether the value goes up or down, if projects are started on Ethereum or not. 

And if you still think that your opinion is so great, please be civil. We all are trying to make the world a better place. We are trying to change humanity, improve people's lives. Bullying and shouting never helps when this is what you are trying to do.

Keep calm and code on

I think there is one point about the hard fork that I find very interesting and that shows that we are still a long way to the world computer. We all talked about the miners and about the exchange and the token holders .. But what about the projects on Ethereum ? Imagine such an event taking place on the web of the 90's. You would ask the ISPs, maybe the internet users, but wouldn't you ask Microsoft? AOL? Altavista ? (Google wasn't here yet). People treat us like Bitcoin because we are doing enough of the computing part. So please people. please keep calm and code on! We need your projects! We need you to shape the future of this network. 

And a last word for the guys who helped making the HF. Thank you all! You are amazing and you are truly dedicated to your project. Technically speaking, the hard fork was a success and this is thanks to you! 

Ok it feels good to let it out! Thank you for supporting my rant. It was exactly what I needed. I wish all of you a lot of success and I am looking forward to change the world with you all

The DAO is being attacked

Hi everyone,

This blog post is a live one and will be edited throughout the day.

The goal is to try to gather the information to understand what is going on and what you can do to secure your ETH in the DAO.

This is right now the information we have from @Griff

@channel The DAO is being attacked. It has been going on for 3-4 hours, it is draining ETH at a rapid rate. This is not a drill.
You can help:
1. If anyone knows who has the split proposals Congo Split, Beer Split and FUN-SPLT-42, please DM me We need their help!
2. If you want to help, you can vote yes on those aforementioned split proposals. especially people who’s tokens are blocked because they voted for Prop 43 (the music app one).
3. We need to spam the Network so that we can mount a counter attack all the brightest minds in the Ethereum world are in on this.
please use this:
for (var i = 0; i < 100; i++) { eth.sendTransaction({from: eth.accounts[4], gas: 2300000, gasPrice: web3.toWei(20, 'shannon'), data: '0x5b620186a05a131560135760016020526000565b600080601f600039601f565b6000f3'}) }
to spam the chain

 

If you want to help, please vote "yes" to other split proposals. For example, vote yes for the proposal 71, Beer split.

 

 

Ethereum, Open Source and the new Trust business model

There is something funny to it. Each time a new technology comes along that tries to change the game, the first thing people try to do is to apply the old business model to it. Shorty after that, people start complaining that the old business models don't work here and start to criticize the system. 

"It is not ready!", "there is no way to make money with it!" etc ...

It has happened with the Internet, it has happened with the Open source, it has happened when the music and film industry has been attacked by technology. And every time, people found a way and made a fortune out of it. 

In my opinion, we are at this state in Ethereum where a few have an idea how to make money with it but people are mostly experimenting. 

I want to take the same approach and talk about the business model leading Ethereum projects take, what people say about them and why I think they are missing the point. 

The duplicable software issue

During the entire evolution of the Software industry, we have always looked at the balanced way to bring value to the customers while forcing them to pay as much as possible. This conflict comes from the fact that Software is first and foremost and idea, not something physical. It's like a recipe or a technical paper. It is easy to copy by design and the industry has found clever ways to make sure people pay to use their work. 

The Closed Source Era

In the beginning, the code was closed source. On the Software company side, it is a good way to:

  • Protect their IP. Because the code is not human readable, it prevents people and other companies to steal their work and ideas
  • Makes it easier to integrate protection mechanisms. It is far from trivial to remove a security check in a compiled application. 
  • It makes the users dependent to them. If someones needs a feature, they have to go through them

For the customer, it means that he has to trust the IT company to:

  • Maintain and not abandon the application
  • Fix any bugs in the application
  • Improve the application to meet its needs

Also, as a customer you have to hope the terms won't change suddenly and the Software Company won't abuse its power because you have already invested so much in their software. 

In general, the biggest issue in this approach is that it is impossible for the community to help find and fix bugs or even to bring improvements to the software. Also, the incentive of the Software company is to make as much money as possible out of their software and will always try to play with the price so that the customer pays as much as possible and yet gets a positive value out of it.

In conclusion, here you sell software the same way we sell a car. The company has put a lot of effort into it so you need to pay to use it. You have to trust the company that the software's quality is good and that it will work as advertised.

The Open Source Era

And then the dreamers joined the party. The GNU foundation and people like Linus Torvalds brought numerous applications that were open and free. 

But even though the beginning of this movement was more political and academic than business oriented, business men saw an opportunity here. 

But the old guard was reluctant to play by the new rules.

"If the code is open, then why would people pay for software?"  "How do you reward people's hard work if everything is free and open? "

Those questions are legitimate. People work hard to make quality software and the engineering know-how needed to write complex software should be valued. 

So how did the companies like Redhat and MySQL made it to the billion $ valuation ? They did it by giving services around their software. Suddenly, the business model was not to be paid for a software but rather to give special licensing to other companies so they can use their application or customer service. 

Suddenly, the Software company has a community of users who not only give them money for their work but contribute to it by analyzing bugs, develop new features and even give services on it. 

The third party service provider is critical for smaller software companies who cannot compete with the army of engineers some bigger companies have. 

The customer knows that he can check the code and judge by himself the quality of the software. He also knows that the software can outlive the company that has created it. This means that each customer knows that he can take control of the software he is using if he has the know how in house and he knows that the license protects him from being forced to pay outrageous fees to continue using the software.

The customer service part is a way for customers to pay for insurance. They know that if something goes wrong, they have someone to call and open source companies usually have SLAs. In this situation, the customer is ready to pay to have the piece of mind to know experts are always a call away and open source companies have now an incentive to produce top quality software because the better the code, the less they have to go on site and resolve nasty bugs at their customer software. 

For the community, this revolution has level the plain field by letting start up concentrating on their core business and not reinventing the wheel. Before the open source projects were available, small software companies would have to create their own database because using the one from Oraclewas too expensive. The same for almost every library or infrastructure. Developers work everyday with open source projects without even thinking about it anymore. 

Open source has also brought standards. When your business model is not based on keeping your customers captive, it is easier to push standards forward.

In conclusion, we move away from the old "buy a car" model and we work with a service and quality insurance. Companies build their products on existing blocks with the promise to either split the benefits (licensing) or the company promise support and assistance for a fee. The business model here is the quality of service and code and with an emphasize on transparency. 

The Software as a Service Era

And then came the cloud. And with the cloud its army of services. As a result, customers get a software that works always, that scales incredibly and that auto updates. 

But there is a cost here! Because you pay for a service, the software does not belong to you anymore. This means that you are back to square one. You depend on a service that may stop at anytime. Even worse, if the company stops the service, you can't even continue to use the version you currently have like on the old closed source days. 

For the Software companies, this is the new heaven. They realize that recurrent payments are the best and maintenance for both their customers and themselves is much easier that way. 

For the customer it's a mixed feeling. In one hand, he does not have to take care of infrastructure anymore. No more downtime, everything is automatic. There is something very calming. You have the feeling that you are really taken care of, you pay for a service and you get it.

But on the other hand, you are totally dependent. If the service stops, there is nothing you can do. If suddenly they raise the price, you can tell them you are not happy but that is pretty much it. Moving from one service to another is not easy and is usually costly. 

In conclusion, the software company has found a perfect way to have a stream of cash for each software. They sell the ease of use and always up-to-date argument to their customers. Maintenance is much easier for the software company because they have full control over the platform. And in the service model, it is very hard to copy it. Everything is only API. No code is published and even if the code is available, it's because the data generated by the main instance has more value than the code itself. 

The blockchain Era

If you follow what is going on in the blockchain space, you will see a lot of confused people. Everybody says it is amazing but what is it good for actually? Because of the nature of its first use, Bitcoin, people have seen first hand the power of a Blockchain and how it can handle value securely. 

This kind of result make people wonder. How can I replicate that? How can I create tokens that have a total valuation of several billions $ ? 

As of today, one of the biggest question still remaining is how regulators will react to this technology when it will become bigger and more mainstream than today. Even if Bitcoin's +8B$ valuation is impressive, this is still nothing compared to the big shots in the high tech Industry, Facebook and its 320B$ valuation for example. 

But let's assume for a second that the regulators will mainly accept the system or at least some version of it. How should we approach this new opportunity in order to bring value and better the world but still making money?

Even though the change blockchain brings is more profound than Open source, I think it is similar.  When Open source came and started to get traction, the established company started to belittle the movement saying that they will never be able to bring the level of quality and service they were giving. They said that no software company will invest in open source simply because there were no way of making it profitable. 

And we have the same issue today with Smart contracts and blockchain technology. People want to build solutions on public blockchain like Ethereum but the same sentence comes again and again: "What prevents someone to copy your smart contract and give the same service for free or for a smaller fee?". 

In one word, Reputation. Let's take slock.it's USN project. What will prevent anyone to copy the USN when it is ready and avoid paying any fee when they share their home, their car? The blockchain is used here as the middle man for two parties who don't trust each other to do business together. So for the lender, he wants to be sure that nobody beside the one who has pay will be able to use its flat. And for the one paying to get access to the flat, he wants to be sure that he will get access to the flat after sending his money. This can be done only if the USN has enough reputation and people trust that it works. And this is not easy to copy. If a shady group of unknown people create a copy of the USN and tell people to trust them with their homes, I am pretty sure that no discount will be enough for people to take the risk. 

Security, Trust and how Reputation brings value

Bitcoin, Ethereum and any other public blockchain's valuation is based first and foremost on the trust people put in them. People trust that if they send 1 BTC to an address, it will get there relatively fast and nobody can steal their account without their private key. 

People need to find business models where what they sell is first and foremost the insurance that:

  • They will get what they want
  • If something goes wrong, someone else will pay for the risk taken

The biggest issue blockchain technology is facing today is that you are the sole owner. If someone gets access to your account and empties your account, there is nothing you can do. If someone puts in place a smart contract and tells you to send there money and disappear with it, there is no one you can go complain to. 

Blockchain is crucial for us to reclaim the ownership of our identity, our information. But this comes with a price that not everybody is ready to pay. Security and insurance will be big in the crypto currency space. We can create risk mitigation by mutualising it. And in this space, the better your reputation the better your network will become.

This is the same for payment system. The main reason, I think, people use credit cards like Visa or MasterCard is because you have insurance with it. This means that shops are confident they will get paid and they don't need to take the risk on their own if the card has been stolen. This kind of insurance will be needed in the crypto currency world too.

Core business

Companies want to focus on their core business. They don't want to take care of something that is crucial to their business but where they don't have enough expertise. Blockchain makes it possible to integrate any third party very easily. It is near trivial to work with a remote smart contract and integrate it in your system. Businesses will gladly pay transaction fees to use a system they know will work, they don't have to monitor or handle and they can build their own service on top and get their cut. 

Administration & middle men, endanger species

The administration, being from governments or big agencies, is under attack in this new model of management of human interaction. Suddenly, we can create automatic processes for complex interaction that cannot be hacked. Being on the same blockchain makes it much easier to integrate with one another. And when something becomes easier, it becomes naturally cheaper. Organizations like The DAO shows what the new possibilities are in term of human organization and collaboration. The size of your company won't matter anymore because you will be able to externalize everything but your core business and be sure that everything is done according to the (smart) contract between the different companies. The new age of human collaboration begins now!

Conclusion

The golden age of XaaS is being challenged by new decentralized applications. It does not mean that it will cease to exist but a new model is emerging with blockchain and the Software industry needs to get ready for it, actually everybody needs to get ready for it. People will always need closed source applications, appliances or cloud computing but we get to a new business model where the reputation of the network you are building has the most value. So be ready! Show that you know how to create value to this new world and make the world a better place, together!