John Liao's Blog

Time 2021-11-14 14:05:05

Web Name: John Liao's Blog

WebSite: http://jyliao.blogspot.com

ID:232337

Keywords:

John,Liao,Blog,

Description:

keywords:
description:
skip to main | skip to sidebarJohn Liao's Blog

Monday, June 24, 2013 Riak CAP Tuning and F#

Riak provides the ability to tune CAP. CAP, which stands for Consistency, Availability, and Partition tolerance, does not seem like controls that are tunable. These terms seem evoke images of binary choices, as in either you have it or you don't. CAP terms by itself is ambiguous in their definitions. I'm not the only one who feels that way as can be seen in Daniel Abadi's blog post. For me, it was more helpful for me to think of tradeoffs as consistency latency (time needed to achieve eventual consistency), performance (read/write latency), and node failure tolerance (how many nodes can fail and still have a working cluster).

Riak exposes their CAP tuning controls via the named variables N, R, and W. These variables are defined as follows:

NNumber of nodes to replicated a piece of dataRNumber of nodes to read data to be considered success (read failure tolerance)WNumber of nodes to write data to be considered write complete (write fault tolerance)

In addition, Riak exposes these additional tuning controls:

PRNumber of primary, non-fallback nodes that must return results for a successful readPWNumber of primary, non-fallback nodes that must accept a writeDWNumber of nodes which have received an acknowledgement of the write from the storage backend

---

Bucket Level CAP Controls in Riak

Here's an example on how to set bucket level CAP settings in Riak with CorrugatedIron:

// Get existing bucket propertieslet properties = ciClient.GetBucketProperties("animals",true).Value// Set # of nodes a write must ultimately replicate to// This should be set at the creation of the bucketproperties.SetNVal(3u)// Set number of nodes that must successfully written before successful write responseproperties.SetWVal(2u)// Set # of nodes required to read a value succesfullyproperties.SetRVal(1u)// Set primary read valueproperties.SetPrVal(1u)// Set primary write valueproperties.SetPwVal(1u)// Set durable write valueproperties.SetDwVal(1u)// Change bucket properties with these new CAP control valuesciClient.SetBucketProperties("animals",properties)

---

Per Request CAP Controls in Riak

Riak allows you to tune CAP controls at per request level:

// Setting W  breed="Cairn Terrier"; score=5})ciClient.Put(data,options)// Get item with R value set to 2ciClient.Get("animals","toto",1u).Value.GetObjectAnimal()// Specify quorumlet getOptions = new RiakGetOptions()getOptions.SetR("quorum")// Need to convert IRiakClient to RiakClient in order to set RiakGetOptionslet client = ciClient :?> RiakClientclient.Get("animals","toto",getOptions).Value.GetObjectAnimal()

Posted byJohn Liaoat6:00 AM52commentsLabels:CAP,f#,fsharp,functional programming,Riak Monday, June 03, 2013 Riak MapReduce with F#

Lately, I have been reading the book Signals and Noise by Nate Silver. His book references an IBM webpagethat claims the world is creating 2.5 quintillion (10¹⁸) bytes of data a day and that 90% of the data that currently exists the world was created in the past two years. Combine this fact with the rise in multicore processors, would explain the surging interest in functional programming and NoSQL and MapReduce frameworks. There is just too much data to be handled by a single machine alone

Continuing the journey in the bookSeven Databases in Seven Weeks as I explore MapReduce. MapReduce framework is popularized by Google's seminal paper, but the genesis of that framework started with ideas in functional programming languages. After all, if functional programming is intended to help one run code on multiple processors on a single server, it's a logical conclusion to extend that computing from multiple processors on a single server to multiple servers. However, it's the recent technology factors and the data tsunami that I believe encourages people to look at functional programming and NoSQL databases.

What I would like to see in the future is that MapReduce capability is integrated with functional programming in such a way that calling a map/reduce function on a typed collection that recognizes that the typed collection is backed by a NoSQL database that it seamlessly spawns the function to the NoSQL stores in a MapReduce fashion.

I want to do something like the following:

Seq.map myfunc myRiakCollection

and the compiler/runtime will automatically convert myfunc into an appropriate form to run on all the data nodes and returns the results. But since we're not there yet, we need to do a little more work.

Some additional items to note, Riak Handbook warns that:

MapReduce in Riak is not exactly the same as Hadoop, where you can practically analyzean infinite amount of data. In Riak, data is pulled out of the system to beanalyzed. That alone sets the boundaries of what's possible.

This is further reinforced by this following mailing list entry in a response from Sean Cribbs, developer advocate from Basho:

...Riak's MapReduce is designed for low-latency queries, but it's also designed for "targeted" queries where you don't slurp the entire contents of a bucket from disk. Structure your data so that you can know -- or deterministically guess -- what keys should be involved in your query. Then you won't be trying to fetch and filter all of the data all the time.

I wanted to point this out as this goes against my mental model of how MapReduce should work.

---

MapReduce with CorrugatedIron

In this particular example, I will use the Javascript example from the book. First, we'll define some helper functions that provides some syntatic sugar in working with the CorrugatedIron's fluent interface:

// Helper function for CorrugatedIron's job inputs for bucket/key pairs only.let setinputs data (q:RiakMapReduceQuery) =     let inputs = new RiakBucketKeyInput()    data    | List.iter( fun (bucket,key) - inputs.AddBucketKey(bucket,key))     q.Inputs(inputs)//  Helper function mapping Javascript functionslet mapjs  src keep (q:RiakMapReduceQuery) =    q.MapJs(fun m - m.Source(src).Keep(keep) | ignore)// Helper function for mapping stored functionslet mapsf bucket func keep (q:RiakMapReduceQuery) =    q.MapJs(fun m - m.BucketKey(bucket,func).Keep(keep) | ignore)// Helper function for mapping builtin functionslet mapbuiltins func keep (q:RiakMapReduceQuery) =    q.MapJs(fun m - m.Name(func).Keep(true) | ignore)// Getting results from CorrugatedIron seems convoluted to me// Helper function is used to extract the data to a simpler form let getResults (results:RiakResultRiakMapReduceResult) =    let getText raw = raw | System.Text.Encoding.Default.GetString    results.Value.PhaseResults  | Seq.map (fun phase - phase.Values | Seq.map getText )  | Seq.concat  // Helper function for Reduce operations in Javascriptlet reducejs src keep (q:RiakMapReduceQuery) =    q.ReduceJs(fun m - m.Source(src).Keep(keep) | ignore)// Helper function for link walking with mapreducelet linkphase bucket keep (q:RiakMapReduceQuery) =    q.Link(fun l - l.Keep(keep).Bucket(bucket) | ignore) 

Here's the script to perform the MapReduce in CorrugatedIrons. In the following snippet, ciClient refers to CorrugatedIron's RiakClient object, in the previous blogs, ciClient was just client:

// Mapreduce example from the booklet src ="function(v) {     /* From the Riak object, pull data and parse it as JSON */     var parsed_data = JSON.parse(v.values[0].data);     var data = {};     /* Key capacity number by room style string */     data[parsed_data.style] = parsed_data.capacity;     return [data];   }"let rooms =  [("rooms","101");("rooms","102");("rooms","103");]let query = new RiakMapReduceQuery()   | setinputs rooms   | mapjs src trueciClient.MapReduce(query)| getResults

Results:

seq ["[{"suite":3}]"; "[{"king":1}]"; "[{"double":4}]"]

---

MapReduce with REST API

It's not all that hard to do MapReduce with ASP.NET MVC REST API although your json script becomes more complex. In the following example, I had to escape the double quote(") because, alas, I'm still using F# 2.0. If you're using F# 3.0, you can use the new triple quote string syntax to wrap the MapReduce script.

// This helper function will be used in all the later examples with REST APIlet mapred script =    let post_url = sprintf "%s/mapred" riakurl     let content = new StringContent(script)    content.Headers.ContentType.MediaType - "application/json"    let response = restClient.PostAsync(post_url,content)    response.Wait()    let results = response.Result.Content.ReadAsStringAsync()    resultslet mapreduceScript =  "{    \"inputs\":[        [\"rooms\",\"101\"],[\"rooms\",\"102\"],[\"rooms\",\"103\"]    ],    \"query\":[        {\"map\":{            \"language\":\"javascript\",            \"source\":            \"function(v) {                /* From the Riak object, pull data and parse it as JSON */                var parsed_data = JSON.parse(v.values[0].data);                var data = {};                /* Key capacity number by room style string */                data[parsed_data.style] = parsed_data.capacity;                return [data];            }\"        }}      ]    }"mapred mapreduceScript 

Results:

val it : Taskstring =  System.Threading.Tasks.Task`1[System.String]    {AsyncState = null;     CreationOptions = None;     Exception = null;     Id = 2;     IsCanceled = false;     IsCompleted = true;     IsFaulted = false;     Result = "[{"king":1},{"suite":3},{"double":4}]";     Status = RanToCompletion;}

---

Stored Functions with CorrugatedIron

I can store custom Javascript mapreduce functions in a specific bucket/key and call it for use:

let myfunc = " function(v) {  var parsed_data = JSON.parse(v.values[0].data);  var data = {};  data[parsed_data.style] = parsed_data.capacity;  return [data]; }"// Store the map function in a bucket valuelet putResults = new RiakObject("my_functions","map_capacity", myfunc)                  | ciClient.Put// Querying using stored map functionlet rooms = [("rooms","101");("rooms","102");("rooms","103");("rooms","104");]let query = new RiakMapReduceQuery()   | setinputs rooms   | mapsf "my_functions" "map_capacity" trueciClient.MapReduce(query)| getResults

Results:

seq ["[{"suite":3}]"; "[{"king":1}]"; "[{"double":4}]"; "[{"suite":7}]"]

---

Stored Functions with REST API

With REST API, reusing the mapred function defined previously:

let storedFuncScript = "{    \"inputs\":[        [\"rooms\",\"101\"],[\"rooms\",\"102\"],[\"rooms\",\"103\"],[\"rooms\",\"104\"]    ],    \"query\":[        {\"map\":{        \"language\":\"javascript\",        \"bucket\":\"my_functions\",        \"key\":\"map_capacity\"    }}    ]}"let savesf bucket key myfunc =    let post_url = sprintf "%s/riak/%s/%s" riakurl bucket key    let content = new StringContent(myfunc)    content.Headers.ContentType.MediaType <- "application/json"    restClient.PostAsync(post_url,content)// Store the map function in a bucket valuesavesf "my_functions" "map_capacity" myfuncmapred storedFuncScript

Results:

val it : TaskTaskstring =  System.Threading.Tasks.Task`1[System.String]    {AsyncState = null;     CreationOptions = None;     Exception = null;     Id = 5;     IsCanceled = false;     IsCompleted = true;     IsFaulted = false;     Result = "[{"suite":3},{"double":4},{"king":1},{"suite":7}]";     Status = RanToCompletion;}

---

Built-in Functions with CorrugatedIron

Riak has some built-in mapreduce javascript functions. I couldn't find any reference documentation on it, the closes I came is the actual Javascript code in github: . Here are the list of builtin Javascript functions from that source code:

Riak.getClassName(obj)Riak.filterNotFound(values)Riak.mapValues(value,keyData,arg)Riak.mapValuesJson(value,keyData,arg)Riak.mapByFields(value,keyData,fields)Riak.reduceSum(values,arg)Riak.reduceMin(values,arg)Riak.reduceMax(values,arg)Riak.reduceSort(value,arg)Riak.reduceNumericSort(value, arg)Riak.reduceLimit(value, arg)Riak.reduceSlice(value, arg)

In the following example, we're only going to use Riak.mapValuesJson():

let rooms = [("rooms","101");("rooms","102");("rooms","103");("rooms","104");]let query = new RiakMapReduceQuery()   | setinputs rooms   | mapbuiltins "Riak.mapValuesJson" truelet results = ciClient.MapReduce(query)// Collect the results and flatten the nested collectionsresults.Value.PhaseResults| Seq.map (fun x - x.GetObjectsIEnumerableRoom())| Seq.concat| Seq.concat

Results:

val it : seqRoom =  seq [{style = "double"; capacity = 4;};        {style = "suite";  capacity = 3;};     {style = "suite";  capacity = 7;};     {style = "king";   capacity = 1;}]

---

Built-in Functions with CorrugatedIron

Calling built-in functions with REST API:

let builtinFunc = "{ \"inputs\":[  [\"rooms\",\"101\"],[\"rooms\",\"102\"],[\"rooms\",\"103\"] ], \"query\":[  {\"map\":{  \"language\":\"javascript\",  \"name\":\"Riak.mapValuesJson\" }} ]}"mapred builtinFunc

Results:

val it : Taskstring =  System.Threading.Tasks.Task`1[System.String]    {AsyncState = null;     CreationOptions = None;     Exception = null;     Id = 6;     IsCanceled = false;     IsCompleted = true;     IsFaulted = false;     Result = "[{"style":"double","capacity":4},             {"style":"king","capacity":1},    {"style":"suite","capacity":3}]";     Status = RanToCompletion;}

---

Reducing with CorrugatedIron

Reduce operations with CorrguatedIron:

let reduceScript = "function(v) {  var totals = {};  for (var i in v) { for(var style in v[i]) {   if (totals[style]) totals[style] += v[i][style];   else               totals[style] = v[i][style];   }  }  return [totals];        }"let query = (new RiakMapReduceQuery()).Inputs("rooms")   | mapsf "my_functions" "map_capacity" false   | reducejs reduceScript trueciClient.MapReduce(query)| getResults

Results:

seq["[{"queen":9051,"king":9189,"single":8811,"double":8629,"suite":9231}]"]

Reduce operations with REST API

MapReduce with REST API:

let myReduceScript = "{    \"inputs\":\"rooms\",    \"query\":[    {\"map\":{        \"language\":\"javascript\",        \"bucket\":\"my_functions\",        \"key\":\"map_capacity\"    }},    {\"reduce\":{        \"language\":\"javascript\",        \"source\":            \"function(v) {                var totals = {};                for (var i in v) {                    for(var style in v[i]) {                        if( totals[style] ) totals[style] += v[i][style];                        else totals[style] = v[i][style];                    }                }                return [totals];            }\"        }}    ]}"mapred myReduceScript

Results:

val it : Taskstring =  System.Threading.Tasks.Task`1[System.String]    {AsyncState = null;     CreationOptions = None;     Exception = null;     Id = 7;     IsCanceled = false;     IsCompleted = true;     IsFaulted = false;     Result = "[{"suite":9231,"king":9189,"double":8629,"queen":9051,"single":8811}]";     Status = RanToCompletion;}

---

Key Filters with CorrugatedIron

Key filters is used to reduce the input set before the map/reduce operations:

let reduceScript = " function(v) {   var totals = {};   for (var i in v) {  for(var style in v[i]) {    if (totals[style]) totals[style] += v[i][style];    else               totals[style] = v[i][style];    }   }   return [totals];        }"let keyFilter = new ActionRiakFluentKeyFilter(fun x - x.StringToInt().LessThan(1000) | ignore)let query = (new RiakMapReduceQuery()).Inputs("rooms").Filter(keyFilter)   | mapsf "my_functions" "map_capacity" false   | reducejs reduceScript trueciClient.MapReduce(query)| getResults

Results:

val it : seqstring =  seq ["[{"queen":780,"suite":968,"king":838,"single":791,"double":758}]"]

---

Key Filters with REST API

Adding key filters with REST API:

let keyFilterScript = "{    \"inputs\":{        \"bucket\":\"rooms\",        \"key_filters\":[[\"string_to_int\"], [\"less_than\", 1000]]},        \"query\":[    {\"map\":{        \"language\":\"javascript\",        \"bucket\":\"my_functions\",        \"key\":\"map_capacity\"    }},    {\"reduce\":{        \"language\":\"javascript\",        \"source\":            \"function(v) {                var totals = {};                for (var i in v) {                    for(var style in v[i]) {                        if( totals[style] ) totals[style] += v[i][style];                        else totals[style] = v[i][style];                    }                }                return [totals];            }\"        }}    ]}"mapred keyFilterScript

Results:

val it : Taskstring =  System.Threading.Tasks.Task`1[System.String]    {AsyncState = null;     CreationOptions = None;     Exception = null;     Id = 10;     IsCanceled = false;     IsCompleted = true;     IsFaulted = false;     Result = "[{"queen":780,"king":838,"suite":968,"single":791,"double":758}]";     Status = RanToCompletion;}

---

MapReduce Link Walking with CorrugatedIron

In my previous blog, I talked about the fact I couldn't figure out how to specify the keep flag in the link walk examples. With map reduce link walking in CorrugatedIron, I now can now specify the keep flag for each of the link-map-reduce phases:

let keyFilter = new ActionRiakFluentKeyFilter(fun x - x.Equal("2") | ignore)let query = (new RiakMapReduceQuery()).Inputs("cages").Filter(keyFilter)   | linkphase "animals" false   | mapjs "function(v) { return [v]; }" trueciClient.MapReduce(query)| getResults

Results:

val it : seqseqstring =  seq    [seq [];     seq ["[{    "bucket":"animals",    "key":"ace",    "vclock":"a85hYGBgymDKBVIcrFrhrwI5O5wzmBKZ8lgZlj/hPM0HlVLm/PczkDP1FlRqHUgqCwA=",    "values":      [{"metadata":     {"X-Riak-VTag":"4XZxrmgvKzan8X4FKz5hti",      "content-type":"application/json",   "index":[],   "X-Riak-Last-Modified":"Thu, 16 May 2013 23:15:58 GMT"},   "data":"{\"nickname\":\"The Wonder Dog\",\"breed\":\"German Shepherd\"}"}]}]"]] 

---

MapReduce Link Walking with REST API

With REST API:

// Code for Link Walking with MapReducelet mrLinkWalk = "{ \"inputs\":{  \"bucket\":\"cages\",  \"key_filters\":[[\"eq\", \"2\"]] }, \"query\":[  {\"link\":{   \"bucket\":\"animals\",   \"keep\":false  }},  {\"map\":{   \"language\":\"javascript\",   \"source\":    \"function(v) { return [v]; }\"  }} ]}"mapred mrLinkWalk

Results:

val it : Taskstring =  System.Threading.Tasks.Task`1[System.String]    {AsyncState = null;     CreationOptions = None;     Exception = null;     Id = 13;     IsCanceled = false;     IsCompleted = true;     IsFaulted = false;     Result = "[{   "bucket":"animals",   "key":"ace",   "vclock":"a85hYGBgymDKBVIcrFrhrwI5O5wzmBKZ8lgZlj/hPM0HlVLm/PczkDP1FlRqHUgqCwA=",   "values":[{     "metadata":{    "X-Riak-VTag":"4XZxrmgvKzan8X4FKz5hti",    "content-type": "application/json",    "index":[],    "X-Riak-Last-Modified":"Thu, 16 May 2013 23:15:58 GMT"},      "data":"{\"nickname\":\"The Wonder Dog\",\"breed\":\"German Shepherd\"}"}]}]";     Status = RanToCompletion;}

Posted byJohn Liaoat6:00 AM2commentsLabels:f#,functional programming,nosql,Riak Sunday, May 12, 2013 Riak Links and Link Walking with F# and CorrugatedIron

Continuing my journey through the book Seven Databases in Seven Weeks, I explore links and link walking in this blog post.Riak has the ability to establish one-way relationship between entries via Links, providing some of the capabilities of a graph database (Riak documentation calls it a lightweight graph database). Riak documentation hints that links should be kept low, on the order of dozens, not thousands. Using the Twitter example from Riak Handbook, you can probably easily find out who Don Syme follows and who does those people follows, etc. But it would be difficult to find all the people that follows Don Syme using Riak's link capability.

---

Adding Links

Here's how you would add links with CorrugatedIron library:

type Cage = { room : int }(*Linking cage 1 to polly via contains, equivalent to doing the followingcurl -X PUT http://localhost:8098/riak/cages/1 \-H "Content-Type: application/json" \-H "Link: /riak/animals/polly; riaktag=\"contains\"" \-d '{"room" : 101}'*)client.Put(  let cage = new RiakObject("cages","1",{room=101})  cage.LinkTo("animals","polly","contains")  cage)(*Putting ace in cage 2 and setting cage 2 next to cage 1.Equivalent to the following sample code:curl -X PUT http://localhost:8091/riak/cages/2 \-H "Content-Type: application/json" \-H "Link:/riak/animals/ace;riaktag=\"contains\",/riak/cages/1;riaktag=\"next_to\"" \-d '{"room" : 101}'*)// Adding more than one link  client.Put(  let cage = new RiakObject("cages","2",{room=101})  [("animals","ace","contains");("cages","1","next_to")]  |> List.iter(fun (bucket,key,tag) -> cage.LinkTo(bucket,key,tag))  cage)  

---

Link Walking

With CorrugatedIron, I can use the API to perform link walking and as an extra bonus, I don't have to worry about extracting data from the multipart/mixed mime types. CorrugateIron library takes care of all of that for us.

(*Link walking, equivalent to curl http://riakhost1:8098/riak/cages/1/_,_,_or in the new version:curl http://riakhost1:8098/buckets/cages/keys/1/_,_,_*) let results = client.WalkLinks(new RiakObject("cages","1"),                               [|new RiakLink(null,null,null)|])         // Dump results, which returns a list of RiakObject(s)results.Value// Since in this particular case, we have only one result, we can do the following// and get back pollyresults.Value.[0].GetObjectAnimal()

Here is the output result for results.Value:

// results.Valueval it : IListRiakObject =  seq    [CorrugatedIron.Models.RiakObject       {BinIndexes = dict [];        Bucket = "animals";        CharSet = null;        ContentEncoding = null;        ContentType = "application/json";        HasChanged = false;        IntIndexes = dict [];        Key = "polly";        LastModified = 1359758822u;        LastModifiedUsec = 523420u;        Links = seq [];        Siblings = seq [];        UserMetaData = dict [];        VTag = "2BTveSKTYDNOZNCiOxyryw";        VTags = seq ["2BTveSKTYDNOZNCiOxyryw"];        Value = [|123uy; 32uy; 34uy; 110uy; 105uy; 99uy; 107uy; 110uy; 97uy;                  109uy; 101uy; 34uy; 32uy; 58uy; 32uy; 34uy; 83uy; 119uy;                  101uy; 101uy; 116uy; 32uy; 80uy; 111uy; 108uy; 108uy; 121uy;                  32uy; 80uy; 117uy; 114uy; 101uy; 98uy; 114uy; 101uy; 100uy;                  34uy; 32uy; 44uy; 32uy; 34uy; 98uy; 114uy; 101uy; 101uy;                  100uy; 34uy; 32uy; 58uy; 32uy; 34uy; 80uy; 117uy; 114uy;                  101uy; 98uy; 114uy; 101uy; 100uy; 34uy; 32uy; 125uy|];        VectorClock = [|107uy; 206uy; 97uy; 96uy; 96uy; 96uy; 204uy; 96uy;                        202uy; 5uy; 82uy; 28uy; 169uy; 111uy; 239uy; 241uy;                        7uy; 114uy; 230uy; 184uy; 103uy; 48uy; 37uy; 50uy;                        230uy; 177uy; 50uy; 60uy; 59uy; 216uy; 112uy; 138uy;                        47uy; 11uy; 0uy|];}]      // results.Value.[0].GetObjectAnimal()val it : Animal = {nickname = "Sweet Polly Purebred";                   breed = "Purebred";}     

There seems to be some limitations with link walking using CorrugatedIron library. There is a WalkLinks() method as part RiakClient, but the input is expecting RiakLink object which has bucket,key, tag as arguments. The link spec is expecting bucket, tag, and keep flag as arguments. I do notice that in the Riak documentation that Link Walking is not available as part of the Protocol Buffers Client (PBC) API, so I'm guessing the WalkLinks() method in CorrugatedIron is either using HTTP protocol or a modified usage of MapReduce. Since link walking is a special case of MapReduce querying, it may not matter much that CorrugatedIron has some limitations on link walking. One other issue with CorrugatedIron and link walking is that when I add multiple links with the same tag and try to link walking with CorrugatedIron, I do not get back all the links, I only get one of the links. In order to follow the examples in the book Seven Databases in Seven Weeks, I can fall back to using ASP.NET MVC Rest API.

Link walking using CorrugatedIron library:

// Link walking, equivalent to // curl http://localhost:8098/riak/cages/2/animals,_,_client.WalkLinks(new RiakObject("cages","2"),                 [|new RiakLink("animals",null,null)|])     // curl http://localhost:8098/riak/cages/2/_,next_to,0/animals,_,_client.WalkLinks(new RiakObject("cages","2"),                 [|new RiakLink(null,null,"next_to");                   new RiakLink("animals",null,null);|])// I can't seem to specify the keep flag with CorrugatedIron, which keeps// intermediate results as you walk beyond primary links// curl http://localhost:8091/riak/cages/2/_,next_to,1/_,_,_     

The book Seven Databases in Seven Weeks is still using the old format for HTTP Link Walking. The new format is as follows:

GET /riak/bucket/key/[bucket],[tag],[keep]            # Old formatGET /buckets/bucket/keys/key/[bucket],[tag],[keep]    # New format

Link walking using REST API:

let riakurl = "http://myriakhost1:8098"let restClient = new HttpClient()type LinkWalkSpec =     { bucket: string; tag: string;  keep: string; }    member x.Link = (sprintf "%s,%s,%s" x.bucket x.tag x.keep)let linkWalker url bucket key (links:LinkWalkSpec list) =    let baseurl = sprintf "%s/buckets/%s/keys/%s" url bucket key    let rec buildLinkWalkUrl (linklist:LinkWalkSpec list) baseUrl =      match linklist with      | [] -> baseUrl      | [x] -> sprintf "%s/%s" baseUrl (x.Link)      | h::t -> let newUrl = sprintf "%s/%s" baseUrl (h.Link)                buildLinkWalkUrl t newUrl    buildLinkWalkUrl links baseurl    |> restClient.GetStringAsync// Equiv to : curl http://localhost:8091/riak/cages/2/_,next_to,1/_,_,_     [{bucket="_";tag="next_to";keep="1"};{bucket="_";tag="_";keep="_"}] |> linkWalker riakurl "cages" "2" 

Link walking results from using REST API:

val it : Taskstring =  System.Threading.Tasks.Task`1[System.String]    {AsyncState = null;     CreationOptions = None;     Exception = null;     Id = 1;     IsCanceled = false;     IsCompleted = false;     IsFaulted = false;     Result = "--CveXyss6PAqBxOOWxeWBCf6eXiiContent-Type: multipart/mixed; boundary=AvYXKrJYDlkeNxh1bQyqDvBAuBF--AvYXKrJYDlkeNxh1bQyqDvBAuBFX-Riak-Vclock: a85hYGBgzGDKBVIcqW/v8Qdy5rhnMCUy5rEy9Oi+P8WXBQA=Location: /buckets/cages/keys/1Content-Type: application/jsonLink: /buckets/animals/keys/polly; riaktag="contains", /buckets/cages; rel="up"Etag: 6gyXgkIgzvwBGRRotqHK3bLast-Modified: Fri, 26 Apr 2013 16:55:40 GMT{"room":101}--AvYXKrJYDlkeNxh1bQyqDvBAuBF----CveXyss6PAqBxOOWxeWBCf6eXiiContent-Type: multipart/mixed; boundary=SaWqmmho48dzhMDmJy3BVcCWrzu--SaWqmmho48dzhMDmJy3BVcCWrzuX-Riak-Vclock: a85hYGBgzGDKBVIcqW/v8Qdy5rhnMCUy5rEyPDvYcIovCwA=Location: /buckets/animals/keys/pollyContent-Type: application/json; charset=utf-8Link: /buckets/animals; rel="up"Etag: 2BTveSKTYDNOZNCiOxyrywLast-Modified: Fri, 01 Feb 2013 22:47:02 GMT{ "nickname" : "Sweet Polly Purebred" , "breed" : "Purebred" }--SaWqmmho48dzhMDmJy3BVcCWrzu----CveXyss6PAqBxOOWxeWBCf6eXii--";     Status = RanToCompletion;}      

Posted byJohn Liaoat9:00 PM47commentsLabels:.NET Functional Programming,f#,fsharp,nosql,Riak Older PostsHomeSubscribe to:Posts (Atom)Search This BlogBlog Archive 2013(12) June(2)Riak CAP Tuning and F#Riak MapReduce with F# May(1) April(2) March(2) February(3) January(2) 2012(12) June(1) May(2) April(2) March(3) February(2) January(2) 2011(3) November(1) May(1) March(1) 2010(2) August(1) May(1) 2009(11) June(2) April(3) March(3) February(2) January(1) 2008(16) December(1) November(1) October(3) September(3) April(1) March(1) February(2) January(4) 2007(18) December(4) November(9) October(1) August(1) July(2) May(1) 2006(2) April(2) 2005(1) October(1)Recommended ReadingProgramming HaskellReal World HaskellProgramming ClojureExpert F# 2.0Subscribe ToPosts Atom PostsAll Comments Atom All CommentsFollowersTopics.NET(9)AppFabric(1)C#(3)Clojure(10)Coherence(1)f#(59)functional programming(44)haskell(4)InfoPath(1)j2ee(1)java(12)Performance(4)perl(1)PowerShell(2)ruby(2)Silverlight(8)Tomcat(1)WCF(1)web services(1)webparts(3)WF(1)WPF(20)xml(1)xslt(1)About MeView my complete profile

TAGS:John Liao Blog 

<<< Thank you for your visit >>>

Hot Websites