<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[CONNECT platform - Medium]]></title>
        <description><![CDATA[Guides on CONNECT platform and general thoughts about asynchronous programming, service composition, future of the cloud, etc. https://github.com/CONNECT-platform - Medium]]></description>
        <link>https://medium.com/connect-platform?source=rss----94cd3f5eb256---4</link>
        <image>
            <url>https://cdn-images-1.medium.com/proxy/1*TGH72Nnw24QL3iV9IOm4VA.png</url>
            <title>CONNECT platform - Medium</title>
            <link>https://medium.com/connect-platform?source=rss----94cd3f5eb256---4</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Thu, 19 Nov 2026 14:09:04 GMT</lastBuildDate>
        <atom:link href="https://medium.com/feed/connect-platform" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com" rel="hub"/>
        <item>
            <title><![CDATA[A Pragmatic Approach to Software Architecture]]></title>
            <link>https://medium.com/connect-platform/a-pragmatic-approach-to-software-architecture-ea51df8a0dd3?source=rss----94cd3f5eb256---4</link>
            <guid isPermaLink="false">https://medium.com/p/ea51df8a0dd3</guid>
            <category><![CDATA[software-architecture]]></category>
            <category><![CDATA[software-development]]></category>
            <category><![CDATA[code-readability]]></category>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[software-enginnering]]></category>
            <dc:creator><![CDATA[Eugene Ghanizadeh]]></dc:creator>
            <pubDate>Tue, 05 Nov 2019 14:45:58 GMT</pubDate>
            <atom:updated>2019-11-06T08:10:50.579Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*UhD46rzfZkfD5Y_9x_HRXg.jpeg" /><figcaption>Illustration of the process of designing software architecture for any particular project. Painting by Adriaen Brouwer [Public domain]</figcaption></figure><h3>Preface</h3><p>The architecture of a software is typically one of the most important aspects of it. It greatly affects its quality, its maintainability, and its reliability. It is also one of the most contested and debated aspects of software engineering in general, typically inciting heated arguments between contributors of a project, arguments that appear to have no potential logical resolve, as the question of <em>“what is a good architecture [for our current project]?” </em>often-times seems as one without a definite answer.</p><p>If you ask experienced software engineers the general question of <em>“what is good software architecture?”</em>, perhaps you will hear something in these lines:</p><blockquote><em>Good software architecture is simple and elegant</em></blockquote><p>(a popular answer which is too vague and subjective to act as a proper basis for objective discussion and decision-making on the matter)</p><blockquote><em>Good software architecture increases maintainability</em></blockquote><p>(a better answer, but still doesn’t help with the question of “how to measure maintainability, aside from hearing in a year or two from programmers working on your architectural design that it is either easy or hard to maintain?”)</p><blockquote><em>Good software architecture incorporates high cohesion and loose coupling</em></blockquote><p>(a much better answer as you can<a href="https://en.wikipedia.org/wiki/Coupling_(computer_programming)"> even precisely measure coupling</a>, but coupling is still an unavoidable phenomenon and this doesn’t give you any clue as how much coupling is acceptable for your case or perhaps how much cohesion is too much cohesion)</p><blockquote><em>Good software architecture properly incorporates established design patterns / make good use of this or that paradigm / etc</em></blockquote><p>(ok established practices, paradigms and patterns do help with coming up with a better design, but how can I know which paradigm suites a project best, or how much should I adhere to a particular pattern?)</p><p>All of these answers are either too subjective, or merely suggest a little bit more objective tools such as paradigms, patterns and metrics without any objective indication of how these tools should be put in use. With this amount of subjectivity at the core of the question of <em>“what is a good architecture”</em>, it is of course no surprise that the discussion on how to design a good architecture for a project easily leads to subjective discords that may never be objectively resolved.</p><p>However, if we take a step back, look at what software architecture actually is and what roles does it play that make it such an integral part of software development, perhaps we can formulate more tangible and quantifiable metrics to base our assessments of various architectural choices and designs on.</p><h3>What is Software Architecture?</h3><p>Generally speaking, the architecture of a particular code-base or project is all of the implicit and explicit rules and conventions guiding how every component is designed and how it talks to other components. These rules can affect any aspect of the project, ranging from paradigms or frameworks it is based on or the abstract layers that it is decomposed to, down to naming conventions, structure of folders and modules, etc. It affects which code elements can be aware of existence of which other code elements or when and how these elements should communicate with each other.</p><h4>What is its purpose?</h4><p>To figure out the purpose of software architecture, we can imagine what happens to a project without one. Assume a project that does not have any agreed upon rules or conventions for modeling and creating various components or dictating how they inter-communicate.</p><p>If multiple people are going to work on this project in parallel, because of that lack of agreed conventions and rules, each one will code their own elements using their own conventions and mental models. This naturally leads to divergent styles, models, patterns and APIs, which not only makes it harder for one contributor to work on the code of another, but also makes it harder for different elements and components to communicate with each other, drastically increasing development time both in the initial phase and for later changes and iterations.</p><h4>So, basically just consistency?</h4><p>Nope, and here lies the distinction between good and bad architecture. Imagine a particular code-base with a particular architecture. Overtime, the project will face some perhaps unpredicted changes and iterations, and facing any such change, one of the following (or a combination) happens:</p><ol><li>The architecture is perfectly in line with intended changes, if not even guiding them. Everything good, nothing to see here.</li><li>The architecture doesn’t say anything about the intended changes. Sometimes developer(s) might discuss, decide and communicate the necessary additions to the architecture with their peers, other times (for example perhaps due to some deadline) they might just implement the change according to models and guidelines inconsistent with the established architecture, contributing to the decline of code-base consistency.</li><li>The developer(s) feel the architecture is in conflict of how the intended change should look like, for example because it adds lots of overhead or it makes it pretty confusing while the change itself is pretty simple and straight-forward. They might choose to respect the architecture and implement the change with much increased cost, or might choose to break pattern, in turn drastically contributing to the decline of code-base consistency.</li></ol><p>In two of these scenarios, there is a good chance that the code-base will be less consistent after the change, and overtime this inconsistency grows to the extent that it looks like the project did not start with an architectural design to begin with. From a pragmatic perspective, we can now just call a good architecture that mostly faces the first case <em>a good architecture</em>, and one that mostly results in the second and third case <em>a bad architecture</em>.</p><h3>Measuring Good Architecture</h3><p>So based on what we discussed above, a good architecture basically should</p><ol><li>Try to account for as many future changes as possible</li><li>Make these changes easier for developers, not harder</li></ol><p>The first item is of-course not really a measurable measure, but it does give us a practical direction to follow: try to envision possible future changes, for example envisioning future phases of the project, parts of the stack that might get swapped out due to external requirements, etc, and see how our architectural decisions and designs would fare facing them.</p><p>The second one however leads to much more quantifiable metrics. On the face of it, it might seem that “easy” and “hard” are subjective terms and there is no proper way of measuring them. However, any change to any codebase is in the end of the day an interaction between a human (the programmer) and a computer (which has a keyboard), and luckily enough we have got a whole field of computer science dedicated to measuring ease or difficulty of such interactions, a field aptly named <strong>Human-Computer Interaction </strong>(or HCI).</p><p>For those who are not familiar with the field, HCI is a field dedicated to analyzing various aspects of various interactions between humans and computers, in as quantifiable a way as possible. Most important amongst these aspects is difficulty (more precisely, the <strong>index of difficulty </strong>of any given interaction). For a lot of basic interactions we’re able to calculate this index, and the fact that we have done so already is why for example the “min-max-close” buttons on the window that you might be reading this on are located at a corner of the screen if you are reading this piece on a laptop or PC (basically the corner causes a huge increase in a parameter that index of difficulty is inversely correlated to). Conveniently enough, index of difficulty also linearly correlates with the time it takes to perform a task (a series of human-computer interactions), which for our use-case can be translated to literal cost of the task itself (literally in terms of money).</p><p>What all of this means? Quite simply put:</p><blockquote>Good architecture reduces the cost of future changes.</blockquote><p>How you can assess that cost for a particular architectural decision? First, imagine some possible changes you might want to make, for example features you might want to add in next phases of the project as mentioned above, or even random changes to the signature of some randomly picked methods/functions/classes. Then estimate the series of interactions that implementing that change might include, from really basic interactions such as how many characters would need to be typed to how many other parts of the code a developer might need to look into to how understandable some particular concept or functionality will be. You can more precisely estimate the difficulty of lower-level interactions (with some basic HCI rules), or even assume arbitrary constants for their difficulty, and ball-park some higher-level interactions (and perhaps take out interactions that are too abstract), and then you’ve got yourself nice estimates of difficulty (and hence cost) of these potential changes.</p><h3>How Will It Look in Practice?</h3><p>Lets take a simplified example and put this approach to practice. For the sake of simplicity, let’s imagine that our developers will not have access to any particular IDE (specifically with nice cross-files searching tools, or other refactoring tools). Imagine the project is structured as follows:</p><pre>src/<br>| -- module-a/<br>| -- | -- index.ts<br>| -- | -- | -- function aOne()<br>| -- | -- | -- function aTwo()<br>| -- | -- a-one.ts<br>| -- | -- | -- function aOneFuncOne()<br>| -- | -- | -- function aOneFuncTwo()<br>| -- | -- a-two.ts<br>| -- | -- | -- function aTwoFuncOne()<br>| -- | -- | -- function aTwoFuncTwo()<br>| -- module-b/<br>| -- | -- index.ts<br>| -- | -- | -- function bOne()<br>| -- | -- | -- function bTwo()<br>| -- | -- b-one.ts<br>| -- | -- | -- function bOneFuncOne()<br>| -- | -- | -- function bOneFuncTwo()<br>| -- | -- b-two.ts<br>| -- | -- | -- function bTwoFuncOne()<br>| -- | -- | -- function bTwoFuncTwo()</pre><p>Now imagine changing of name or signature of src/module-a/a-one/aOneFuncOne() . Without any established rules, I would need to check the body of eleven other functions to see if they use aOneFuncOne() and if they need to change. Note that the difficulty of this check is not the same for functions of src/module-b/b-one compared to functions of src/module-a/a-one , since functions of a-one.ts are already in the same file as aOneFuncOne() . Listing the interactions, for the former I just have:</p><ul><li>read the body of aOneFuncTwo() , see if changes are required, do them if they are</li></ul><p>For the latter, the list of interactions is like this:</p><ul><li>list all submodules of src/</li><li>list all files in src/module-b/</li><li>open src/module-b/b-one.ts</li><li>read the body both bOneFuncOne() and bOneFuncTwo() , see if changes are required, do them if they are</li></ul><p>These interactions each take some time (and cognitive effort), since they each involve at least one click (and perhaps some scrolling and some typing). Similarly, checking other files in src/module-a is easier than checking files in src/module-b , as it requires fewer raw interactions, though it is more difficult than just checking other functions in src/module-a/a-one.ts , as then it would require more interactions.</p><p>Now if the architecture enforced the rule that files in modules other than src/module-a can only use functions defined in src/module-a/index.ts , the difficulty of the change would be reduced drastically. This of-course would have the downside that if some day I need aOneFuncOne() somewhere in module-b , I would need to also change module-a/index.ts to comply with that rule, an overhead whose probability and interaction cost we can again estimate pretty precisely.</p><p>I could even go one step further and require src/module-a/a-one.ts to explicitly mention the functions that it <em>exports</em> to other modules (as Typescript would require this of you regardless of your architectural design), then I could check if aOneFuncOne() is an exported function, and if it is not, the change would again be drastically cheaper to make.</p><p>Similarly, if every other file had to explicitly import aOneFuncOne() via an explicit import statement (i.e. import { aOneFuncOne } from &#39;src/module-a/a-one ), I would have a much easier time checking which functions I need to change. Now if I had another (unmentioned file) with that import command at the beginning, which had 10 other functions within it but only one of them actually using aOneFuncOne() , I would be better off putting that one outlier in its own file, as it would reduce the number of interactions required for detecting and making the change again.</p><p>Note that putting all of my 12 functions in src/module-a/a-one.ts would also not be a good idea, since though it would reduce the difficulty of checking all other functions, it is still a sub-par solution to checking perhaps one or five other function bodies.</p><p>With all of this in mind, we can now make much more objective judgements when deciding between two architectural designs, depending on how inter-dependent my functions would be in each design, and how close or far these inter-dependent functions would be structured. Note that this is just another way to say we should prefer an architecture that is <em>more cohesive and loosely coupled</em>, but this time around we have more tangible metrics to assess how much more cohesion or loose-coupling are we talking about.</p><h3>What about readability/intuitiveness?</h3><p>In our example, we glossed over a key factor in calculating difficulty of some of the interactions: in reality, the difficulty of reading and navigating some code is not constant. It is instead greatly affected by readability of the code (locally) and intuitiveness of the underlying model of the architecture (on a larger scale). A stream of random characters is much harder to read through and find something in, and an unintuitive structure turns the interaction of navigating the code-base into a blind and highly error-prone brute-force, as opposed to easy and quick navigation you would get with an intuitive understanding of the underlying model.</p><p>Fortunately, for readability, there are <a href="https://sarahfakhoury.com/2019-icpc-readability.pdf">good analytic studies</a> of how to measure and improve them, alongside a plethora of more <a href="https://www.nngroup.com/articles/legibility-readability-comprehension/">generic HCI studies</a> that can be useful on the matter (<a href="https://www.researchgate.net/publication/4284058_Human_Computer_Interaction_Legibility_and_Contrast">some even useful to find-out most productive editor color-themes</a>).</p><p>Intuitiveness might seem more elusive as it definitely is more subjective. However, we only need to consider intuitiveness as in <em>“the architecture makes sense”</em> so that it reduces cost of changes, which in turn means <em>“the architecture makes sense to people who are going to make those changes”</em>. Besides this, we also know that <em>something</em> makes more sense to <em>someone</em> (or at least takes much less time and effort on their part to understand it) if (and almost but not always only-if) it relates to what they already know and/or have experienced, with more weight given to more repeated experiences or more recent experiences. Combining these two facts gives us a much more objective perspective on intuitive architectural design:</p><blockquote>An architecture that more closely resembles what your team already knows and has experience in is more intuitive.</blockquote><p>Note that an implication of this fact is that the best architectural design and choices for a particular project <strong>IS NOT</strong> just determined by the project itself, but also by the team that is going to work on it. If your choice of architectural style is one that has some noticeable learning curve (a famous example of which is the famous MVC), then it might be a really good choice for a team already familiar with the style, but not so much for a team without such an experience. Furthermore, if the future team composition changes from one with the aforementioned experience to one mostly without it, you MUST consider the cost of refactoring the whole architecture accordingly compared to the learning cost that each new team member will have.</p><h3>Take Aways</h3><h4>This example was too simplistic, will this work IRL?</h4><p>Although our example was relatively abstract and simplistic in nature, it touches one of the most essential aspects of software architecture: coupling and cohesion. More elaborate and complex examples mostly boil down to the same considerations and optimizations in the end of the day (alongside perhaps intuitiveness of the model for the problem domain). For example, MVC architectures break code-bases into three layers on the assumption that code in one layer (for example view) is more dependent on other code in the same layer (other view logic) than code in other layers (for example, the model layer). This is for example true when your view logic is highly dependent on some rendering framework/library, more than it is dependent on the controller layer logic, and the separation drastically reduces the cost of changes such as swapping out the rendering framework/library (which is why it was a more popular pattern at a time that such a change was more probable).</p><h4>Does this mean I no longer need to rely on established paradigms and patterns?</h4><p>Nope. This is an approach for objectively and quantitatively analyzing various architectural designs / decisions / even patterns and paradigms, not coming up with them. If the architecture of a software is the problem (<em>“With which rules should I design my code-base to reduce the cost of future change?”</em>), then various architectural paradigms and patterns offer solutions / solution-templates for that problem, and the aforementioned difficulty-measuring approach allows you to analyze and compare these solutions.</p><p>Furthermore, the established concepts, patterns and paradigms ARE solutions designed with the same considerations (directly or indirectly). As discussed above, MVC (or any other layer-based architecture) tends to contain most probable changes to a particular layer to reduce its potential cost, and cohesion and coupling are basically emergent from change-difficulty optimization, as we saw above. However, this approach allows you to properly verify which patterns and paradigms are most applicable to your case, and which might actually cause more overhead in the end of the day.</p><p>I realize that in the end of the day, software architecture still is a really hotly debated topic among software engineers. A lot of zealotry goes around with respect to strongly held opinion of what an ideal software architecture is, not deeming programmers not sharing these opinions worthy of the title of programmer to begin-with. The sentiment, even among more flexible engineers is that for each problem (hence project), there is a unique, beautifully elegant, borderline work-of-art architectural design that is future-proof until the end of time. But I fear most of the time these radical notions stem from the fact that people are not in touch with the pragmatic reasons of why we need a good architecture for a project, let alone what that means. However, if we drop these irrational biases and look at software architecture in the most detached and pragmatic manner possible, it all becomes quite clear:</p><blockquote>Good architecture reduces the cost of future changes, for the people making those changes.</blockquote><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ea51df8a0dd3" width="1" height="1" alt=""><hr><p><a href="https://medium.com/connect-platform/a-pragmatic-approach-to-software-architecture-ea51df8a0dd3">A Pragmatic Approach to Software Architecture</a> was originally published in <a href="https://medium.com/connect-platform">CONNECT platform</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[RxJS: what debounce and throttle really do]]></title>
            <link>https://medium.com/connect-platform/rxjs-what-debounce-and-throttle-really-do-b70e39b772ff?source=rss----94cd3f5eb256---4</link>
            <guid isPermaLink="false">https://medium.com/p/b70e39b772ff</guid>
            <category><![CDATA[reactivex]]></category>
            <category><![CDATA[reactive-programming]]></category>
            <category><![CDATA[rxjs]]></category>
            <category><![CDATA[javascript]]></category>
            <category><![CDATA[typescript]]></category>
            <dc:creator><![CDATA[Eugene Ghanizadeh]]></dc:creator>
            <pubDate>Wed, 26 Jun 2019 21:08:10 GMT</pubDate>
            <atom:updated>2019-06-26T21:08:10.734Z</atom:updated>
            <content:encoded><![CDATA[<p>P.S. <a href="https://medium.com/u/fa05e2bb3f0d">ReactiveExtensions*</a> please document your stuff properly</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*N8Aa8KChjqTwIXwem-AJjA.png" /></figure><p>Recently I was reading up on <a href="https://rxjs-dev.firebaseapp.com">RxJS</a>’s debounce() and throttle() operators, trying to figure out how they exactly work. The typical place to check this information is this nice website called learnrxjs.io, which in this case gives this definition for debounce():</p><blockquote>Discard emitted values that take less than the specified time, based on selector function, between output.</blockquote><p>and the following for throttle() :</p><blockquote>Emit value on the leading edge of an interval, but suppress new values until durationSelector has completed.</blockquote><p>Both description seem to indicate that you somehow need to specify some sort of time-span or interval for these operators, however, a look at the examples provided in the same site shows that this is indeed wrong:</p><pre>const debouncedExample = example.pipe(debounce(() =&gt; timer(1000)));</pre><pre>const throttleExample = source.pipe(throttle(val =&gt; interval(2000)));</pre><p>As you can see in these examples, it is the internal Observables that care about “time” and “interval”s, not debounce() and throttle() themselves. This made me curious to see what exactly would happen if I passed some other non-timer-related Observables to these operators. As it turned out, neither debounce() nor throttle() have anything to do with timers and intervals:</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fround-lake.dustinice.workers.dev%3A443%2Fhttps%2Fstackblitz.com%2Fedit%2Faoisdunasldinlasld-axhuhn%3Fdevtoolsheight%3D1000%26embed%3D1%26file%3Dindex.ts%26hideExplorer%3D1%26hideNavigation%3D1&amp;url=https%3A%2F%2Fround-lake.dustinice.workers.dev%3A443%2Fhttps%2Fstackblitz.com%2Fedit%2Faoisdunasldinlasld-axhuhn%3Fembed%3D1%26file%3Dindex.ts%26hideExplorer%3D1%26hideNavigation%3D1%26devtoolsheight%3D1000&amp;image=https%3A%2F%2Fround-lake.dustinice.workers.dev%3A443%2Fhttps%2Fc.staticblitz.com%2Fassets%2Ficon-664493542621427cc8adae5e8f50d632f87aaa6ea1ce5b01e9a3d05b57940a9f.png&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=stackblitz" width="745" height="400" frameborder="0" scrolling="no"><a href="https://medium.com/media/6a1aabc3117448049e8a7fda5b3e97db/href">https://medium.com/media/6a1aabc3117448049e8a7fda5b3e97db/href</a></iframe><p>As it turns out, debounce() actually creates a <strong>control observable</strong> for every value that the source emits, using the factory function you provide it with. If the source emits another value before this control observable completes, it will simply throw out the control observable, alongside its corresponding value, and create a new control observable for the newly emitted value. However if the control observable completes before the source emits again, then that value will be emitted.</p><p>In other words, debounce() will create one Observable per value emitted by the source, and will only emit that value if that Observable completes earlier than the next emission from the source. The only exception to this rule is when the source itself completes, in which case its last emitted value will also be emitted by debounce() .</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fround-lake.dustinice.workers.dev%3A443%2Fhttps%2Fstackblitz.com%2Fedit%2Faoisdunasldinlasld-usrvx9%3Fdevtoolsheight%3D1000%26embed%3D1%26file%3Dindex.ts%26hideExplorer%3D1%26hideNavigation%3D1&amp;url=https%3A%2F%2Fround-lake.dustinice.workers.dev%3A443%2Fhttps%2Fstackblitz.com%2Fedit%2Faoisdunasldinlasld-usrvx9%3Fembed%3D1%26file%3Dindex.ts%26hideExplorer%3D1%26hideNavigation%3D1%26devtoolsheight%3D1000&amp;image=https%3A%2F%2Fround-lake.dustinice.workers.dev%3A443%2Fhttps%2Fc.staticblitz.com%2Fassets%2Ficon-664493542621427cc8adae5e8f50d632f87aaa6ea1ce5b01e9a3d05b57940a9f.png&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=stackblitz" width="745" height="400" frameborder="0" scrolling="no"><a href="https://medium.com/media/49f655bab41c8306133aad7626934659/href">https://medium.com/media/49f655bab41c8306133aad7626934659/href</a></iframe><p>Similarly, throttle() creates a control observable for a value emitted by the source. After that, it will ignore all other emissions by the source until that control observable completes, after which it will emit the original value that the control observable was created for. Note that this also means that no control observables are created for the ignored values. Lastly, similar to debounce() , it will emit the last value of the source when the source completes.</p><p>Fun Fact: None of these snippets will work if you remove the setImmediate()s, or if you put all the emissions in the same function body. I suspect that’s due to some sort of race condition which will cause the values to be ignored, but either way it simply means that if you try to cook up some synchronous process using debounce() or throttle(), you will most probably fail as you will find them not behaving as you would expect.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b70e39b772ff" width="1" height="1" alt=""><hr><p><a href="https://medium.com/connect-platform/rxjs-what-debounce-and-throttle-really-do-b70e39b772ff">RxJS: what debounce and throttle really do</a> was originally published in <a href="https://medium.com/connect-platform">CONNECT platform</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[WRITE-UP: Adding A Web-Based Remote-Shell to a PaaS]]></title>
            <link>https://medium.com/connect-platform/write-up-adding-a-web-based-remote-shell-to-a-paas-bd1e5f88d2b2?source=rss----94cd3f5eb256---4</link>
            <guid isPermaLink="false">https://medium.com/p/bd1e5f88d2b2</guid>
            <category><![CDATA[security]]></category>
            <category><![CDATA[docker]]></category>
            <category><![CDATA[shell]]></category>
            <category><![CDATA[nodejs]]></category>
            <category><![CDATA[connect-platform]]></category>
            <dc:creator><![CDATA[Eugene Ghanizadeh]]></dc:creator>
            <pubDate>Wed, 05 Jun 2019 15:18:27 GMT</pubDate>
            <atom:updated>2019-06-05T15:18:27.288Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*fFIRyliB-l9qQkuFxCU7Zg.png" /></figure><p><strong><em>DISCLAIMER:</em></strong><em> This is not a guide. The scope of the problem and the technical challenges and solutions involved might feel pretty specific to a lot of programmers and thus seem irrelevant. However, I personally enjoy such in-depth technical stories and would enjoy more prominent existence of such write-ups, which is why I’m writing this. If you feel the same, also join the band and write and share similar stories of fun technical journeys of yours.</em></p><h3>What I Wanted to Do</h3><h4>Background</h4><p><a href="https://connect-platform.com">CONNECT-platform</a> is a project that I’ve been working on for some time now, the core is a free and open-source NodeJs framework built on top of <a href="https://expressjs.com">Express</a> that enables visual programming without limiting capabilities of NodeJs (you can <a href="https://medium.com/connect-platform/the-problem-of-async-programming-and-a-crazy-idea-for-solving-it-cf368a9ea949">read here more about the intentions behind it here</a>). We then expanded this core to a Platform as a Service (PaaS), where you can get ready made instances of NodeJs running CONNECT-platform on the fly, in your browser (fun fact is that we built the whole PaaS using the framework itself to ensure that it is pretty good even in really complicated scenarios, and it took us less than a week to put the first version of the PaaS on AWS this way).</p><p>The way that works is simple:</p><ul><li>The default <a href="https://www.npmjs.com/package/connect-platform">NPM package</a> hosts the framework and the Visual Code Editor,</li><li>There is a <a href="https://github.com/CONNECT-platform/connect-platform-boilerplate">boilerplate project on github</a> that runs the framework with basic necessary configuration (also reading necessary config from environment variables),</li><li>There is the <a href="https://cloud.docker.com/u/loreanvictor/repository/docker/loreanvictor/connect-platform">docker image</a> that clones that boilerplate project and does all the setup necessary,</li><li>The PaaS itself runs a container from that image for each project, feeding necessary environment variables to it,</li><li>Finally, an <a href="http://openresty.org/en/">OpenResty</a> server distributes requests to each instance.</li></ul><h4>Goal</h4><p>In the setup described above, the Visual Code Editor is basically the only interface one will have for controlling their projects on the PaaS. As a developer this felt pretty sub-optimal for me, as besides a code-editor, the rest of the main stuff that I do with a coding project involve a shell. For example, I conduct version control of my projects with shell, I do log reading with shell, etc.</p><p>The solution? Obviously web-based remote shell access to any instance on CONNECT-platform. And by <em>shell access</em> I do not mean simply running bash instructions on the instance (to be honest that was already available as the Visual Editor would give me full JS-powered control over the Node VM running on each instance), but to run a full terminal emulator capable of interactive commands (such as using Vim to edit files).</p><p>Security-wise: someone with access to the panel (Visual Editor) of each instance basically already could do whatever shell access could do, so the corner-stone for the security design of this remote shell should have been to fully reduce its security problem to the security problem of the panel itself, i.e. access should be granted to the shell only through the panel, and the shell should have exactly the same access-level in the container as the Node process.</p><h3>How It Went</h3><h4>Part 1: The Remote Shell Daemon</h4><p>Initially I was not fully aware of the difference between full-blown interactive shell emulation and just sending bash commands and receiving the results on the front-end, so I started with looking for an easy-to-integrate front-end solution that I could attach to the existing Visual Editor (the panel). A quick google search got me to <a href="https://xtermjs.org">Xterm.js</a>, which looks like the de-facto front-end tool for that. I checked out some of the projects using Xterm.js and quickly realized that what I needed was complete terminal emulation, so I basically had to have a process on each instance doing that job for me as well, ideally working nicely with Xterm.js on its own too.</p><p>The first candidate for that was <a href="https://github.com/krishnasrinivas/wetty">WeTTy</a>, simply because it seemed like I could just run it on my existing Node process in the container. However the <a href="https://github.com/krishnasrinivas/wetty/tree/master/docs">API doc</a> (or the <a href="https://github.com/krishnasrinivas/wetty/blob/master/README.md">README</a>) were really not helping on how to do attach a WeTTy server on an existing Express server, and further-more a simple test proved that it couldn’t be done seamlessly without some minimum level of configuration (<a href="https://repl.it/repls/BeneficialFocusedCommand">you can check it out on this repl.it piece</a>). As a super-lazy developer, this caused me to simple drop WeTTy as a candidate here, with the prospect of not having a separate process for the remote-shell besides the Node process.</p><p>I found other options, most prominently <a href="https://github.com/yudai/gotty">GoTTy</a> and <a href="https://github.com/tsl0922/ttyd">ttyd</a>. Both seemed really similar and capable of getting the job done, and both also provide the Shell-Client as well (so no need to wrestle with Xterm.js on client-side). However not only ttyd has instructions on how to install on linux systems, it has a proper Dockerfile alongside one specifically for alpine images, which is the base image for CONNECT-platform containers as well. Hence, it was the obvious winner.</p><h4>Part 2: Security Layer</h4><p>So to have a web-based remote-shell, I would be running a ttyd service alongside each instance of CONNECT platform. ttyd would listen on some port and offer the whatever I was looking for. However, the security concerns mentioned above would not be satisfied if I was to just run the ttyd within each container and expose it to the internet. To recap, these are the security criteria I wanted to meet:</p><ul><li>The remote-shell should have identical access-level to the Node VM, i.e. something can be done by the shell if and only if it can be done by the NodeVM</li><li>Access to the remote-shell is identical to access to the panel, i.e. someone can access the remote-shell if and only if they can access the panel.</li></ul><p>The first criteria could easily be solved by ensuring that the NodeVM spawns the ttyd process as a child process:</p><pre>const { exec } = require(‘child_process’);</pre><pre>const run = () =&gt; {<br>  let proc = exec(&#39;ttyd bash&#39;);<br>  process.on(&#39;exit&#39;, () =&gt; proc.kill());<br>}</pre><p>the extra-line here is to ensure that when the Node process is dead (for whatever reason), so would be the ttyd process.</p><p>I could have alternatively just ensured that the process for the shell is executed by a user (or user group) with identical access level. However, that would mean that anyone making subsequent changes to the access level of the Node process should also be careful to update the access level of the remote-shell, which would increase the chance of error and hence unnoticed security flaws.</p><p>As for the second constraint, I decided to seal-off the ttyd server within the container and only give the Node VM access to it. Since the panel has a secure line to the Node VM, the Node VM could then authenticate and authorize requests to access the shell originating from the panel, similar to how it would authenticate and authorize other requests from the panel, and if authorized, proxy the requests to the ttyd process. As with the first constraint, I could keep the two services completely separate and ensure identical authorization mechanism on a higher-level, however again that would mean that any subsequent changes to auth process of one of the services would need to follow corresponding changes to auth process of the other, again, increasing possibility of error.</p><h4>Part 3: Proxying ttyd</h4><p>So the security design meant that there should be an authentication mechanism known only to ttyd and the Node process in the first place. ttyd supports basic HTTP authentication, so it would suffice to create a random user-name and password on each execution and feed it to the process:</p><pre>const run = (credentials) =&gt; {<br>  let proc = exec(`ttyd -c ${credentials} bash`);<br>  process.on(&#39;exit&#39;, () =&gt; proc.kill();<br>}</pre><p>and in the main guy supposed to run this whole charade:</p><pre>let credentials = randomToken() + &#39;:&#39; + randomToken();<br>run(credentials);</pre><p>Now I needed to track what are the requests that the ttyd client makes to its server, which of those are authenticated. For these requests then I would need to:</p><ol><li>Authenticate the incoming request through Panel’s auth process,</li><li>Modify the proxied request to add the basic authentication HTTP header according to the generated credentials,</li><li>Proxy the request</li></ol><p>As it turned out upon further inspection, the requests in need of authentication are the first original request to get the shell, and another request for a file named /auth_token.js , which seems to be used to establish and authenticate a WebSocket with the ttyd server.</p><p>As a lazy developer, this meant that I would need a simple proxy tool working nice with Express also properly capable of handling WebSockets. As it turned out with another quick googling, <a href="https://github.com/chimurai/http-proxy-middleware">http-proxy-middleware</a> is just that. So the proxying code would look something like this:</p><pre>const proxy = require(&#39;http-proxy-middleware&#39;);</pre><pre>const SHELL_URL = &#39;&lt;local address for ttyd&gt;&#39;;</pre><pre>//<br>// this will be the url that the shell will be accessible on <br>// publicly, e.g. if the shell is to be accessed via:<br>// <a href="https://project.connect-platform.com/shell/">https://whatever.connect-platform.com/shell</a><br>// then TTY_PATH would be &#39;/shell&#39;<br>//<br>const TTY_PATH = &#39;&lt;some url sub-path&gt;&#39;;</pre><pre>const doproxy = credentials =&gt; {<br>  return proxy([TTY_PATH, &#39;/auth_token.js&#39;],<br>    {<br>      target: SHELL_URL,  // so we send everything to ttyd,<br>      ws: true,           // so we proxy WebSockets,<br>      changeOrigin: true, // so that ttyd is not aware of proxying<br>      pathRewrite: path =&gt; {<br>        //<br>        // well, ttyd expects requests to reach it on &#39;/&#39;, so<br>        // we should properly trim them.<br>        //<br>        if (path.startsWith(TTY_PATH))<br>          return path.substr(TTY_PATH.length);<br>        else return path;<br>      },<br>      onProxyReq: (proxyReq, req) =&gt; {<br>        if (auth(req)) { // check if the request is authenticated,<br>          //<br>          // well we need to set &#39;Authorization&#39;<br>          // header on requests, with credentials<br>          // in base64 format.<br>          //<br>          proxyReq.setHeader(<br>            &#39;Authorization&#39;,<br>            &#39;Basic &#39; + new Buffer(credentials).toString(&#39;base64&#39;)<br>          );<br>        }<br>      }<br>    }<br>  );<br>}</pre><p>With the main code being modified to look like this:</p><pre>const main = app =&gt; {<br>  let credentials = randomToken() + &#39;:&#39; + randomToken();<br>  run(credentials);<br>  app.use(doproxy(credentials));<br>}</pre><p>As for authenticating the requests, since the request to the root path of the shell ( TTY_PATH ) originate from panel itself, and since panel’s authorization is done with verifying the proper access token, I could just inject the token to the URL on panel’s side, and read and verify it here. In other words, the initial request to access the shell would look like this:</p><pre>https://whatever.connect-platform.com/&lt;TTY_PATH&gt;?token=&lt;the_token&gt;</pre><p>so it could be easily checked like this:</p><pre>const auth = req =&gt; {<br>  if (req.originalUrl.startsWith(TTY_PATH))<br>    return req.query.token &amp;&amp; verify(req.query.token);<br>  return false;<br>}</pre><p>Note that the verify() function here can be any form of token verification. On CONNECT-platform, we use <a href="https://jwt.io">JWT</a> (which <a href="https://www.npmjs.com/package/jsonwebtoken">also has a pretty nice Node package</a>).</p><p>However, the other request to /auth_token.js is made by the ttyd client itself (which I suspect is possible to modify, but again, lazy developer here). However, the referrer on that request must still be the original request to TTY_PATH , which means the token must reside in the referrer and can be verified similarly. Which means the auth() function would turn into this:</p><pre>const { URL } = require(&#39;url&#39;);</pre><pre>const auth = req =&gt; {<br>  if (req.originalUrl.startsWith(TTY_PATH)) {<br>    return req.query.token &amp;&amp; verify(req.query.token);<br>  }<br>  else if (req.originalUrl == &#39;/auth_token.js&#39;) {</pre><pre>    const referrer = new URL(req.get(&#39;Referer&#39;));<br>    return referrer.searchParams.has(&#39;token&#39;) &amp;&amp;<br>           verify(referrer.searchParams.get(&#39;token&#39;));</pre><pre>  }</pre><pre>  return false;<br>}</pre><h4>Part 4: Configuring OpenResty</h4><p>So now we have a ttyd running on the container, providing the Shell-Client as well, while also being fully concealed by Node VM and secured (or at least, exactly as much as the Node VM). The task seemed to be mostly done. I made the whole remote-shell (and its access path) configurable, in case that ttyd is not even installed in the environment that the platform is running in (because remember, the core of the platform is a NPM package that can be run anywhere). I also added a bit more tracing of the ttyd process to the JS code, so that if it would detect errors in executing the ttyd process, it would not share the link to the remote-shell with the client.</p><p>Subsequently, I added the proper piece of code to the boilerplate project to read configuration of the remote-shell from the environment variables, and added the proper environment variables to the Dockerfile. This is also where the alpine Dockerfile of ttyd came in REAL handy, as I just added the minimal content to the Dockerfile of the platform itself.</p><p>Now everything seemed nicely working, so I published the new version of the NPM package, pushed to the boilerplate repo’s master and re-built and pushed the latest version of the Docker image. I asked my colleague to update the Docker image used on the PaaS, and sat back thinking the task was done.</p><p>Obviously though, it wasn’t. As it turned out, It seemed like the WebSocket could not conduct a proper handshake. As it turned out, the handshake process is initially a simple HTTP(S) request bearing an Upgrade and a Connection header with values indicating that this request should be “Upgraded” to an open-tunnel for the WebSocket. Apparently, NGINX (and by extension, OpenResty) does not set these headers by default, so you should ask it explicitly to do so, simply by adding the following to its config:</p><pre>proxy_http_version 1.1;<br>proxy_set_header Upgrade $http_upgrade;<br>proxy_set_header Connection &quot;Upgrade&quot;;</pre><p>The http version is just to ensure that the proper version of the protocol is used for establishing a WebSocket connection.</p><h3>The End</h3><p>So in the end, <a href="https://connect-platform.com">CONNECT-platform</a> was equipped with a pretty dope and relatively safe remote-shell. I was actually surprised by how relatively fast and smooth the whole process went (it took me 1.5 days in the end), considering that I had really vague idea of many concepts involved (from the shell itself, to the proper proxying mechanism or how WebSockets work precisely).</p><p>All the code changes mentioned in this article can be found in the repos that are linked. The bulk of the code changes on the platform can be found <a href="https://github.com/CONNECT-platform/connect-platform/tree/master/platform/bind/shell">here</a>, in case you are curious to inspect it more closely.</p><p>Anyways, this whole technical write-up thing is something new that I am experimenting with. This is something that I would personally love to find and read more online, thats why I thought of sharing my own fun technical experiences. I would be really glad to hear what you think, both the actual technical challenge or .Also special bonus for mentioning sources and blogs bearing similar tech write-ups.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=bd1e5f88d2b2" width="1" height="1" alt=""><hr><p><a href="https://medium.com/connect-platform/write-up-adding-a-web-based-remote-shell-to-a-paas-bd1e5f88d2b2">WRITE-UP: Adding A Web-Based Remote-Shell to a PaaS</a> was originally published in <a href="https://medium.com/connect-platform">CONNECT platform</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How To Write a CONNECT platform Package]]></title>
            <link>https://medium.com/connect-platform/how-to-write-a-connect-platform-package-b903bb9d9fbb?source=rss----94cd3f5eb256---4</link>
            <guid isPermaLink="false">https://medium.com/p/b903bb9d9fbb</guid>
            <category><![CDATA[nodejs]]></category>
            <category><![CDATA[javascript]]></category>
            <category><![CDATA[npm]]></category>
            <category><![CDATA[connect-platform]]></category>
            <category><![CDATA[package]]></category>
            <dc:creator><![CDATA[Eugene Ghanizadeh]]></dc:creator>
            <pubDate>Tue, 02 Apr 2019 20:36:05 GMT</pubDate>
            <atom:updated>2019-04-02T20:36:05.320Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*nqKUxtTQq_lRzui0NJWv_A.png" /></figure><p>CONNECT platform packages are simple NodeJS packages that allow you to expand what you (and your fellow developers) can do with <a href="https://connect-platform.com">CONNECT platform</a>. As discussed <a href="https://medium.com/connect-platform/inline-coding-in-connect-platform-534fce3c8cdf">here</a>, you can already add any Javascript code snippet to your platform if need be, and that ability, combined with the fact that the platform utilizes NPM as its package-manager under the hood, allows you to utilize any functionality of any NPM package on your instances.</p><p>However, it might be a neater idea to wrap a Javascript module to create a proper CONNECT platform package for more convenience. In this guide we explore how you can do that, utilizing the boilerplate project for creating a CONNECT platform package.</p><h3>Setup</h3><p>1 Clone the <a href="https://github.com/CONNECT-platform/connect-platform-package-boilerplate">CONNECT platform package boilerplate</a>, and rename the folder to what you want your package to be named:</p><pre>&gt; git clone <a href="https://github.com/loreanvictor/connect-platform-package-boilerplate.git">https://github.com/loreanvictor/connect-platform-package-boilerplate.git</a></pre><pre>&gt; mv connect-platform-package-boilerplate tutorial-package</pre><p>2 Add a git repository (this step is obviously not necessary, but its smart to do this). I will create a public GitHub repo for this tutorial:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/1*0LwsVjXdWNA_xtkNn_z4mw.gif" /></figure><p>It is <em>highly recommended to </em><strong><em>NOT</em></strong><em> initialize a README file or any other files</em>, as the boilerplate package has everything necessary. Now you need to change the repo of the cloned code to the one you just created:</p><pre>&gt; cd tutorial-package<br>&gt; git remote set-url origin <a href="https://github.com/loreanvictor/tutorial-package.git">https://github.com/loreanvictor/tutorial-package.git</a><br>&gt; git push</pre><p>which should cause your GitHub repo to look like this (on refresh):</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*kX8_6d9H1ye0_3g5GvU0kw.png" /></figure><p>3 Update your package.json file. Update the &quot;name&quot; , &quot;description&quot; , &quot;repository&quot; , &quot;author&quot; and the &quot;bugs&quot; fields. You might also want to update &quot;version&quot; and perhaps &quot;license&quot; if you want to use a different license for your package. In the end, it should be updated to look something like this:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/e4091311ced76efb1867c778bc5e68fe/href">https://medium.com/media/e4091311ced76efb1867c778bc5e68fe/href</a></iframe><p>It is also a good idea to recreate your package-lock.json :</p><pre>&gt; rm -f package-lock.json<br>&gt; npm i</pre><p>If you are a good open-source samaritan perhaps you should update README.md as well.</p><h3>Development</h3><p>So now that you have your setup ready, lets discuss how you can start developing a package.</p><p>As mentioned earlier, CONNECT packages are nothing but simple Javascript modules, with some additional flavor. On the most basic level, a package is a Javascript code that registers some platform Nodes (if you have no idea what Nodes are, maybe check <a href="https://medium.com/connect-platform/connect-platform-basic-concepts-543b79c3094e">this doc</a> first). For example, this code snippet does that in the simplest possible manner:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/880f1c256f6e791cd62ab2bbbcece1f7/href">https://medium.com/media/880f1c256f6e791cd62ab2bbbcece1f7/href</a></iframe><p>This simply registers a Node on path /my-package/hellow that gets a name as an input and returns a greeting to that name. You can take a look at hellow.js in your boilerplate setup for much more in-depth information, for example on what options you have for the <em>signature</em> object (the first parameter in platform.node() call in the code snippet above) register your function with.</p><p>You can register any function using platform.node() . The function you register will be invoked whenever there is a <em>Call </em>to the path you register it on. If you provide a function f() , then it will be called like this:</p><pre>f(inputs, output, control, error);</pre><p>Where</p><ul><li>inputs is a dictionary of all given inputs,</li><li>output is a function that you can call to send an output. It takes two arguments, first the name of the output, and the second its data. The name of the output should match one of the strings in &quot;outputs&quot; field of the <em>signature</em> object.</li><li>control is a function similar to output, but for activating control outputs instead of normal outputs (or to send signals instead of data as a result of the execution of your Node, for example a simple “success” flag without additional data). It gets one input, which is the name of the control output to be activated, and it should match one of the strings in &quot;controlOutputs&quot; field of the <em>signature</em> object.</li><li>error is a function that you can use to throw errors. It is really important to use this function instead of just throwing an error specifically when you have asynchronous calls, as although the platform will try its best to catch all errors that might stem from your code, it might not be able to catch the ones resulted in asynchronous calls. So make sure you have proper error handling (sync or async) in your code and that you pass the caught errors to this function (or you can simply pass a string indicating the nature of the error).</li></ul><p>That is basically it. The boilerplate package you already cloned is a simple “hellow world” package with lots of comments to explain everything in code, and you can also check out <a href="https://github.com/loreanvictor/connect-jwt">this</a> or <a href="https://github.com/loreanvictor/connect-firestore">this</a> package as for more complex examples.</p><h4>But How To Test It?</h4><p>The package boilerplate installs connect-platform as a dev dependency and configures it so you can easily test the Nodes you add in the platform. Simply run this:</p><pre>&gt; npm start</pre><p>and checkout localhost:4000/panel to see your test area. For example, you can test the Node described in hellow.js like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/1*dldKyZtg056bF__N_ZMGwA.gif" /></figure><h4>Package Configuration</h4><p>Perhaps you have noticed that the hellow.js in the boilerplate code is actually not the entry point of your package (in package.json ). Although in theory, you could make a file like that and set it as the entry point of your package, which would actually function as a proper CONNECT platform package, it is almost always a better idea to add some configuration info to also tell the platform how to load your package and where everything belongs.</p><p>If you take a look at the actual entry point of your boilerplate package, which is index.js , you will see something like this:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/558e3df12099db02b498d5a425bcd15b/href">https://medium.com/media/558e3df12099db02b498d5a425bcd15b/href</a></iframe><p>Which basically should explain how to configure your package :) Just a few notes:</p><ul><li>The &quot;nodes&quot;.&quot;native&quot; field in your configuration will basically tell the platform to simply require the given files. It will of course search in a different order compared to require() ‘s default behavior, prioritizing the root of your package. It will also map which Nodes are loaded by each package.</li><li>You can also load Nodes that you have created by the platform itself, which are stored in <em>JSON</em> format, using the &quot;nodes&quot;.&quot;json&quot; field in your configuration. Except from, well, parsing the JSON file, this behaves exactly as native modules.</li></ul><h3>Publishing</h3><p>If you want to make it accessible through NPM for all other developers like you, simply do this:</p><pre>&gt; npm publish</pre><p><em>(Note that you don’t need to do that to be able to access a package you have made on your own CONNECT instances. CONNECT platform utilizes NPM to install packages, so you can easily install any Git repository as well, even private ones).</em></p><p>Now everyone can install your package by going to the “Packages” section of their platform, click on “Install From NPM” and enter the name of your package:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/1*jYybS7eyVPm7bbTcHOvCow.gif" /></figure><p>If you want your package to appear on the first screen, looking gloriously with its icon and description, then you need to create a pull-request for the <a href="https://github.com/CONNECT-platform/connect-repo">CONNECT platform’s package repository</a>, which is an open-source project as well. Simply update the <a href="https://github.com/CONNECT-platform/connect-repo/blob/master/repo.json">repo.json</a> file, by copying another one of the packages and modifying it to reflect your own package, and make a pull-request. The moderators will take a look and approve your package.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b903bb9d9fbb" width="1" height="1" alt=""><hr><p><a href="https://medium.com/connect-platform/how-to-write-a-connect-platform-package-b903bb9d9fbb">How To Write a CONNECT platform Package</a> was originally published in <a href="https://medium.com/connect-platform">CONNECT platform</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The Problem Of Async Programming, And A Crazy Idea For Solving It]]></title>
            <link>https://medium.com/connect-platform/the-problem-of-async-programming-and-a-crazy-idea-for-solving-it-cf368a9ea949?source=rss----94cd3f5eb256---4</link>
            <guid isPermaLink="false">https://medium.com/p/cf368a9ea949</guid>
            <category><![CDATA[javascript]]></category>
            <category><![CDATA[callback-hell]]></category>
            <category><![CDATA[asynchronous]]></category>
            <category><![CDATA[nodejs]]></category>
            <category><![CDATA[backend-development]]></category>
            <dc:creator><![CDATA[Eugene Ghanizadeh]]></dc:creator>
            <pubDate>Mon, 10 Dec 2018 21:10:11 GMT</pubDate>
            <atom:updated>2018-12-11T01:05:08.934Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*GGTVHrKjs8421dDO5CZNtQ.png" /><figcaption>an image inducing the idea of breaking some basic rules of traditional programming in order to move forward to the next generation</figcaption></figure><h3>The Problem</h3><p>If you have done any non-trivial asynchronous programming recently, for example if you’ve done some nice backend development using NodeJS, you have quickly came face to face with a serious problem infesting the beloved programming paradigm: your codes quickly turn into hardly readable and much harder to debug tangles of code pieces (usually wrapped inside anonymous functions) that connect to each other in a specifically mind-boggling way. In other words, you most definitely have either generated or was forced to work on code infested with a <strong>callback hell</strong>:</p><pre>Promise.all([<br>  doA().then(aOut =&gt;<br>    new Promise(resolve =&gt; {<br>      doC(aOut).then(cOut =&gt; {<br>        resolve([aOut, cOut]);<br>      });<br>    })),<br>  doB().then(bOut =&gt;<br>    new Promise(resolve =&gt; {<br>      doD(bOut);<br>      resolve(bOut);<br>    }))<br>]).then(values =&gt; {<br>  doE(values[0][1], values[0][0], values[1]);<br>});</pre><p><strong>Callback hell</strong>s are a pretty easy to spot, yet they are rampant and often seem unavoidable. No one seems to be fond of callback hells, so it is safe to assume that no one would create a callback hell on purpose, which implies that they are merely a manifestation of an underlying cause. But if that assumption is true, what would such a cause be?</p><p>Well, <strong>callback hell</strong>s do not infest good old synchronous code at all, so we can simply try to pinpoint their cause by taking a deeper look at the core difference between a sync and an async code:</p><pre>let aOut = doA();<br>let bOut = doB();<br>let cOut = doC(aOut);<br>doD(bOut);<br>doE(cOut, aOut, bOut);</pre><p>This is a synchronous counter-part for the aforementioned async code. It yields the same results, however, the key difference lies in its execution order (and subsequently performance):</p><p>in the sync code, doC() is executed after doA() and doB() are complete, although it just needed doA() to complete and not doB(). Similarly, doD() waits unnecessarily for doA() and doC() to finish as well, and doE() awaits doD() while it does not need to.</p><p>The reason for these undesired <em>“waits”</em> in the sync code is simple: sync code is executed sequentially. Statements in a sync piece of code follow each other as they are executed one after another exactly in the order they are written. If we were to describe our sync code in plain English, we would say something like <em>“doA() then doB() then doC() then doD() then doE()”</em>, and if we were to represent this flow, using a box representing each statement and an arrow representing each <em>“then”</em> word, we would get something like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*KNoyuMY1TK8r3ClYX1oOLQ.png" /></figure><p>Since our code-based representation is text-based, it inherently is equipped with an <em>“order”</em> for the statements, i.e. which statement is written in text after which one, so the <em>“arrows”</em> become implicit and redundant, and we omit them from our codes.</p><p>Similarly, our async code in English would be in lines of <em>“doA() then doC(), simultaneously doB() and then doD(), and after doA(), doB() and doC() are done, doE()”</em> which in terms of <em>“boxes”</em> and <em>“arrows”</em> would translate to:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*07QToB-gNjNhoUwGmpUCbw.png" /></figure><p>Looking at this representation, the problem with our async code instantly becomes evident: There is an inherent order to the statements (simply because they are written down and each one have to follow another one) that in the case of the async code is no longer our execution order, so we need extra indicators (like Promise.then callbacks in the code or <em>“then”</em>-arrows in the figure above) to explicitly indicate that separate order. In other words, an essential aspect of our <em>“code”</em> (that it is an ordered sequence of statements) loses its meaning, and we need to explicitly communicate that meaning by other means, which as you can see, results in much extra complexity and hardship.</p><p>Note that it is not just that the order of statements is not aligned with the execution order, as if that was the case, a simple shuffling of our <em>“statement”</em> boxes would have solved the problem. The problem is that our execution flow is no longer a <em>“sequence”</em>, which means we cannot effectively represent it using any <em>“sequence”</em>, namely the sequence in which the statements are written down.</p><p>So does this mean that the problem is that async code is inherently more complex and so the <strong>callback hell </strong>problem can never be solved? By no means. Although async code is inherently more complex, it does not imply that we can never solve the <strong>callback hell</strong> problem. In fact, a lot of solutions have already been proposed and implemented in various languages and frameworks, so its worth taking a look at the most prominent ones to analyze if they truly tackle the problem.</p><h3>Promises</h3><p><strong>Promise</strong>s where the first solution introduced to Javascript community as a solution for the aforementioned problem. They were initially offered as part of some packages, and were so cherished by the community that got adopted as part of the programming language itself. Basically, async codes used to look like this:</p><pre>doA((err, aOut) =&gt; {<br>  // ...<br>}</pre><p>This structure resulted in terribly messy codes, as a statement simply invoking an async function would include all of the callback chain, i.e. you could not single out function invocations without having to read through all of the flow. Additionally, there are many async flows that you simply cannot describe with this structure. For example, you can try to re-write our initial async code in terms of callback arguments and see how it works out.</p><p>However, <strong>Promise</strong>s are not able to truly solve the problem as they are not even truly tackling it. Our initial example of a <strong>callback hell</strong> was not written in terms of callback arguments but actually in terms of Promises. Comparing to our graphs, Promises are basically a much neater way to represent the <em>“arrows”</em>, so they naturally cannot do anything to help with the complex tangle we have got on the graph itself.</p><h3>Async/Await</h3><p>async/await is the newest attempt to solve this problem (at least for Javascript). The way it works is pretty simple: await simply enforces the rest of the body of a function to <em>“wait” </em>the statement to be finished, with the catch that the whole function is now required to be marked with async , meaning that this is an asynchronous function.</p><p>In practice, this gives us extremely similar flow graphs as with the fully sync code, since just like the sync code the statements following await would need to <em>“await”</em> its execution. This was the very same case we had with the key differentiation between our async code and its sync counter-part. The only difference being that the whole context now becomes async, which allows the underlying engine to do other stuff in the mean time.</p><p>In other words, async/await is also not designed to tackle the problem, its just a tool to alleviate the problem when the execution flow itself is actually a <em>“sequence”</em>. So while this does help with simpler situations, again it would not be useful anywhere that real async flow is to be developed. To see that for yourself, try to re-write our async code snippet again using only async/await statements and see how does that pan out (EDIT: which is actually possible, thanks to “<a href="https://news.ycombinator.com/user?id=jmull">jmull</a>” for a pretty neat solution for that).</p><h3>The Crazy Idea</h3><p>Ok since none of the aforementioned solutions are targeting the core of the problem with async code, lets get to the crazy idea that might solve it. Just to re-iterate, our problem was with an async flow graph being a tangle on its own (which naturally manifests in form of <strong>callback hell</strong> tangles, <strong>promise-then</strong> tangles, etc.):</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*07QToB-gNjNhoUwGmpUCbw.png" /></figure><p>Look closely at this problematic graph, and you will notice that the tangle of <em>“arrow”</em> connections is not an inherent problem of this graph, but actually a phenomenon emerging from our representation of choice. In other words, it is not that the graph we are trying to represent is inherently super complicated, its just <em>the way that we insist on representing it makes it complicated</em>. We can simply throw out the requirement for the <em>“box”</em>es to be drawn in a sequential manner, and we can re-arrange this graph like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Bvh714vczFzC0oPeNA4C9g.png" /></figure><p>Simple, isn’t it? :)</p><p>Well, you might be wondering by this point, that OK that looks like a simpler representation of our asynchronous execution flow, however it was the result of throwing the <em>“sequentiality”</em> of our representation out of the window, which is not something that you could do in practice, since, well, <em>any code is a sequence of statements</em> in the end. And you would be right.</p><p>And this is why it is a crazy idea: what if we didn’t code in form of <em>a sequence of statements</em> then, but actually in form of a graph like this one? i.e. what if we replaced <strong>sequential</strong> codes with <strong>graphical</strong> ones for inherently asynchronous codes?</p><p>Well, I asked myself that similar question about a year ago, and since the inherent problems of asynchronous programming, at least the way we do it currently, was constantly bugging me and slowing me down on a daily basis, I started slowly developing a prototype for that on late hours maybe one or two nights each week. Over the course of a year, and with the help and encouragement of some friends, that prototype grew into <a href="https://connect-platform.com">CONNECT</a>, a platform that is built exactly just to solve the aforementioned problem, and basically looks like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*iipP6WTNQs1ktgEy_l2wRQ.png" /></figure><p>Of course, <a href="https://connect-platform.com">CONNECT</a> is not built as a platform for general-purpose asynchronous coding, but as one targeting specifically backend development, and even more specifically logical micro-services in the backend. Why backend? Because front-end async codes are much lighter on their asynchronicity, and also involve a lot of rendering logic, which makes the problem essentially a different one. Why logical? because again, computational backend code is generally not that asynchronous and treating it as such would actually slow you further down. And why micro-services? Well basically because they are the future of backend development.</p><p>Additionally, doing this we realized that backend logic is not purely asynchronous on all levels, and more often it looks like synchronous blocks connected to each other in an asynchronous manner. Fortunately, we could easily factor this into <a href="https://connect-platform.com">CONNECT</a>, as it was already built on top of NodeJS/ExpressJS and did support Javascript expressions seamlessly within a graph. <a href="https://medium.com/connect-platform/inline-coding-in-connect-platform-534fce3c8cdf">We simply allowed each node to become a complete synchronous block on its own</a>.</p><h3>Concerns, Pros &amp; Cons</h3><h4>Concern #1: Visual Programming is not programming</h4><p>I do realize that traditionally programmers (including myself) are against <em>“visual”</em> programming, as it has almost always been something targeting <em>“less-knowledgable”</em> people, and generally has always been a result of trading <em>“fine-control”</em> with <em>“simplicity”</em>, a trade-off that to put it mildly at least annoys any serious enough programmer.</p><p>However, that is not the case for <a href="https://connect-platform.com">CONNECT</a> (or generally the graphic-based coding approach for async flows), as actually it offers more fine-control compared to callback arguments or async/await statements, and offers at least the same amount of control as Promises do. In fact, if anything, sucn solutions enable us to do much more with the asynchronous flows that we develop. This is not giving away control, but rather simply dropping a constraint that is basically holding us back, although that constraint might be a property of any code that we’ve done for a long time and so we might be really used to it.</p><h4>Concern #2: The application of such a tool would be limited</h4><p>Well, since this approach does not remove control from developers and merely offers a more optimal representation, it is not just the case that stuff that you could build with such a tool (such as <a href="https://connect-platform.com">CONNECT</a>) would be more limited, but actually you would become able to build much more complex async logics much easier.</p><p>To actually ensure that, we even built the <a href="https://deskfront.connect-platform.com/auth">Platform as a Service</a> of CONNECT using <a href="https://connect-platform.com">CONNECT</a> itself. Not only did it not impede us by any means, but we were able to accomplish it in a surprisingly fast time-span (4 days to a week, from the moment that we registered the domain and decided the names of various involved micro-services until it was ready to launch CONNECT instances for any registered user, put them to sleep after minutes of non-activity and wake them back up in response to requests).</p><p>Of course, it is notable that since this is an approach to asynchronous programming. There are other contexts of programming that will definitely not benefit from such an approach at all, just the same way that they would not benefit from Promise s or async/await statements.</p><h4>Pro #1: Efficiency</h4><p>Since this approach is designed on making the flow representation much more efficient and hurdle-free, it should naturally result in lower cognitive load on reading/writing the said representation, which would result in much faster development time (and hence much lower development costs). In other words, writing or reading this graph</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Bvh714vczFzC0oPeNA4C9g.png" /></figure><p>would obviously be much easier and hence much less time consuming than this one:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*07QToB-gNjNhoUwGmpUCbw.png" /></figure><h4>Pro #2: Maintainability</h4><p>This one is a natural result of the former. As it becomes much easier to read and follow the flow of the code, it also becomes much easier to debug it or make changes to it, which is one of the main hurdles of our current approaches to async programming.</p><p>In fact, the process becomes so seamless that we realized we could actually easily record the execution of graphs in <a href="https://connect-platform.com">CONNECT</a> and replay them in real-time, not just helping greatly with testing and debugging, but also with performance inspection:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/1*vNvoQSnTbE29zsvhhg7NAQ.gif" /></figure><h4>Pro #3: Performance</h4><p>Since this approach to async programming actually enables much finer control over your asynchronous flow, it avails much better orchestration of tasks that can be accomplished in parallel to reduce response time. While you can do this with traditional approaches as well, since managing any non-trivial asynchronous flow would increase costs of development and maintainability, in many cases you would compromise some performance to keep those costs low. However, this graph-based approach you are enabled to delve into such optimizations without fear of increasing development/maintenance costs.</p><h4>Con #1: Async Flows are inherently different</h4><p>Since one of the first projects with <a href="https://connect-platform.com">CONNECT</a> was a <a href="https://deskfront.connect-platform.com/auth">Platform as a Service</a>, we quickly stumbled upon this one: now that async flows that we would usually do have became much easy to manage, we found ourselves quickly building much more complex flows and graphs (that we would have definitely avoided otherwise as their text-based representations would be too much to handle), which were accordingly challenging for us to understand and work with.</p><p>This also came up in pretty early user-feedbacks we collected: Most people are accustomed to thinking synchronously of their codes and services, and for a good reason: in a synchronous context, you can know for sure exactly which statement is being executed at each step, a control which you naturally loose by going the async route. And while these kinds of challenges are usually hidden behind the headache of callback hells, when we resolved those headaches, they surfaced and we quickly realized that generally you would need a different mental approach to your more complex async graphs than what you would to your sequential codes.</p><p>However, as per our admittedly pretty limited tests, it does not take so long for people to adopt the newly required cognitive approach, although there definitely is a learning curve involved here.</p><h4>Con #2: Tooling</h4><p>For <a href="https://connect-platform.com">CONNECT</a>, we tried to re-use as much of the already available tools and technologies available to us and just add the graph-based representation that we intended. It is based on NodeJS, which means all NPM packages are available for it (which is one of the largest package repositories available out there). We even created a human-friendly JSON-based format for the logical graphs, so that you could easily version-control them (for example using git).</p><p>Despite all this, a pretty early on feedback we got from one of our pilot users was that he was asking for removal of some of the graph data so that he could read the git-diffs in an easier manner. In this case, although the human-friendly format helps a lot, still reading a graph from a JSON-file is just as efficient as forcing an asynchronous flow into a sequential format, it will inevitably be more challenging than usual. Although we are planning on adding a lot of git-focused nice features (including pretty diffs) for <a href="https://connect-platform.com">CONNECT</a> flow graphs, it remains a fact that you would loose availability of development tools that have been for years developed assuming they are to aid with text-based codes, and some of them would need to be developed from the scratch for graph-based codes.</p><h3>Conclusion</h3><p>Async programming is infested with code-tangles and callback hells, simply because being <strong>async</strong> means having a <strong>non-linear non-sequential execution</strong> flow, which is at odds with the sequential nature of text-based codes. The suggested solution here is to go for graph-based codes, in which you explicitly and optimally outline the execution flow which might as well be between blocks of synchronous codes.</p><p>Do you think this radical approach would work? Well there is no need to speculate as we have built a <a href="https://connect-platform.com/">platform</a> that just enables you to test it out for yourself. Give it a try, and share your thoughts and feedback.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=cf368a9ea949" width="1" height="1" alt=""><hr><p><a href="https://medium.com/connect-platform/the-problem-of-async-programming-and-a-crazy-idea-for-solving-it-cf368a9ea949">The Problem Of Async Programming, And A Crazy Idea For Solving It</a> was originally published in <a href="https://medium.com/connect-platform">CONNECT platform</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[CONNECT 102: An Authentication Service]]></title>
            <link>https://medium.com/connect-platform/connect-102-an-authentication-service-402532d3486f?source=rss----94cd3f5eb256---4</link>
            <guid isPermaLink="false">https://medium.com/p/402532d3486f</guid>
            <category><![CDATA[guides-and-tutorials]]></category>
            <category><![CDATA[backend-development]]></category>
            <category><![CDATA[connect-platform]]></category>
            <category><![CDATA[low-code-development]]></category>
            <category><![CDATA[authentication]]></category>
            <dc:creator><![CDATA[Eugene Ghanizadeh]]></dc:creator>
            <pubDate>Sat, 08 Dec 2018 13:13:44 GMT</pubDate>
            <atom:updated>2019-01-20T18:47:48.784Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*nqKUxtTQq_lRzui0NJWv_A.png" /></figure><p>In this tutorial, we will build a more practical backend-service with CONNECT and get introduced to some more nuanced details of the platform. The service we will build is a password-based authentication service.</p><p>I am assuming that you already know how to setup a CONNECT instance. If you do not know how to do that, don’t worry though: checkout <a href="https://connect-platform.com">connect-platform.com </a>and you can create one with one-click.</p><p>I am also assuming that you know the core concepts of CONNECT platform. If not, take a look at <a href="https://medium.com/connect-platform/connect-platform-basic-concepts-543b79c3094e">this article</a> first, and then come back.</p><h3>STEP 0: Setting Up a Database</h3><p>First, we need a database to store user names and passwords. For this tutorial, we will use <a href="https://firebase.google.com/docs/firestore/">Google’s Firestore</a> database, which is really easy to setup and use and is free as well. Go there, and setup a new project:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/1*8ELd3FIm81RwK6R_8RQc9Q.gif" /><figcaption><strong>fig. 1</strong>, creating a new project on Google’s firebase</figcaption></figure><p>Then, add a database to your project:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/1*49cxZOIVsIYwlQIQM-bcfw.gif" /><figcaption><strong>fig. 2</strong>, creating a Firestore database</figcaption></figure><h3>STEP 1: Connecting to Database</h3><p>Now we need to configure our CONNECT instance to enable it to talk to our database. To do that, go to the <strong>PACKAGES</strong> section of your panel, click on the circular plus button, find <strong>connect-firestore</strong> and press install:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/1*pCYyXbN7B4cBdy5ynimJXQ.gif" /><figcaption><strong>fig. 3</strong>, how to install a package</figcaption></figure><p>Now we need to configure our instance to allow it to talk to our Database (obviously not everyone is allowed to talk to our database). If you click on the <strong>connect-firestore</strong> package in your packages list, you’ll see a “how to setup” button which guides you through doing just that. Basically we first need to generate a key for our database:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/1*R4AQiKAfjI8VQG-dmRa0XA.gif" /><figcaption><strong>fig. 4</strong>, generating a key for Firestore</figcaption></figure><p>Now we need to give this key to our CONNECT instance. Go to the <strong>VAULT</strong> section of your panel and add a new key, naming it firestore.json :</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/1*tzIaOflTFSlTKZtKC8migA.gif" /><figcaption><strong>fig. 5</strong>, adding a key to vault</figcaption></figure><p>For the content of the key, open the key-file you just downloaded from Firestore, and copy the contents.</p><p>Finally, we need to state in our project config that our package should use the key we just created. Go to the <strong>CONFIG </strong>section of your panel, and add the following to the JSON:</p><pre>&quot;firestore&quot;: {<br>  &quot;project&quot;: &quot;connect-auth-tutorial&quot;,<br>  &quot;keyfile&quot;: &quot;secure/firestore.json&quot;<br>}</pre><p>Which will make your configuration look like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*0AEbf_ejJ-PkROfmuCQC_Q.png" /><figcaption><strong>fig. 6</strong></figcaption></figure><p>Note that the project name should be the name of your project (you can see it under project-id in your keyfile).</p><h3>STEP 2: Installing Other Packages</h3><p>For a password-based authentication service, besides a database, we also need to be able to encrypt passwords and match encrypted passwords. For this purpose, go back to <strong>PACKAGES</strong> section and install <strong>connect-passwords</strong> package (which is actually a CONNECT integration for bcryptjs ).</p><p>Besides encryption, we also need to be able to create authentication tokens once a user has been authenticated. These tokens will be then used to access the resources of the user without requiring a password every time. For this purpose, <strong>connect-jwt </strong>is a really nice package to install.</p><p>So by this point, our <strong>PACKAGES</strong> section should look like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*V3YEhC_Kp4_bb0l911xFcA.png" /><figcaption><strong>fig. 7</strong>, packages required for password-based authentication</figcaption></figure><p>We also need to have a secret that we will use to sign the aforementioned tokens, so we know that no one else can create fraudulent tokens. For this, just add the following to the <strong>CONFIG</strong>:</p><pre>&quot;secret&quot;: &quot;some-random-string-that-is-secret&quot;</pre><p>Do not forget that the config is in <a href="https://en.wikipedia.org/wiki/JSON">JSON format</a>, so you would need to have commas between various elements in it.</p><h3>STEP 3: Registering Users</h3><p>First, we need to create a node to register users. We should get a username and a password and create a new record in our database. For testing our service, lets also add an pet_name that will only be accessible by the authentication tokens we generate:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*8dQlCXrXO8XFz5v-RWa76A.png" /><figcaption><strong>fig. 8</strong>, the registration node</figcaption></figure><p>As you can see, we are inserting into the collection &quot;users&quot; , and we are packing the username , the password and the pet_name to pass to the /db/insert node. Press the <strong>Test</strong> button to test this out for yourself:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/1*vNvoQSnTbE29zsvhhg7NAQ.gif" /><figcaption><strong>fig. 9</strong>, testing the registration node</figcaption></figure><p>When you run a test, it will ask you for missing inputs as you can see. It will then run your code and record the execution flow and play it for you. You can also use &lt;space&gt; key to play and pause and&lt;left&gt; and &lt;right&gt; arrow keys to jump to previous and next important events.</p><p>You can check the end-result in your Firestore panel:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*g0eU56XafEjOAeXcl-mGQA.png" /><figcaption><strong>fig. 10</strong></figcaption></figure><p>However, note that we have not encrypted the password here. So, lets change our node to this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*6al4k1y95vdP_yxOeNPXAw.png" /><figcaption><strong>fig. 11</strong>, hasing the password</figcaption></figure><p>If you get an error testing the modified version, with the /password/hash box glowing red, its because you have entered a numeric password last time. Click on the password input, and change it to a string one like &quot;1234&quot; .</p><p>Now when you run this, you’ll realize that although the password is now encrypted, we have two users with the username dude in our database:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3i78jAXFX_QCxJHaudx1Yg.png" /><figcaption><strong>fig. 12</strong>, duplicate usernames in database</figcaption></figure><p>To prevent that, first lets add a node that would help us find a user by its username:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*NXlOaE3NGTi6FIazTGaNIg.png" /><figcaption><strong>fig. 13</strong>, finding a user by username</figcaption></figure><p>This one seems like a complex node, however it does a pretty simple thing: It first creates a query object on the database from collection users , filters that query based on equality of the field &quot;username&quot; to the given username input, resolves the query, then checks if the result contains any element (its length is larger than zero). If so, it will return the first object in the list, and if not, a control output of not_found will be invoked.</p><p>Now, lets update our /register/ node:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*C9mRi89L6uzSjhJ4QrSW2A.png" /><figcaption><strong>fig. 14</strong>, checking if a user exists before creating one</figcaption></figure><p>Note that the <strong>Switch Node</strong> with ... inside it means for any value for user, it will invoke the already_exists control output of our registration node.</p><p>Our registration node complete, we should now create one node for logging in using a username and password and another one for accessing the pet_name of a specific user with a given authentication token. Before we proceed, remember to remove the first faulty user record from the database:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/1*_4KH8mYjVkPBmwVB0Aks9g.gif" /><figcaption><strong>fig. 15</strong>, deleting the bogus Firestore record</figcaption></figure><h3>STEP 4: Logging Users In</h3><p>For logging users in, we need a node to get the username and the password as inputs, check if a user with that username exists and if the password matches, and then sign a token for future access to this user’s resources, in our case their pet_name . Such a node would look like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*iA3kvtc081ML63Ftyi5r4Q.png" /><figcaption><strong>fig. 16</strong>, a login node</figcaption></figure><p>Note that /jwt/sign node receives a payload and a key, and we have used the secret we defined earlier in the instance’s config for the key. Also note that for the payload being signed, we are simply putting the username in an object.</p><p>One additional important observation is that we are receiving a raw password for login, however we have stored an encrypted version, and as a result we cannot simply match them with user.password == password . Instead, the <strong>connect-passwords</strong> package also provides /passwords/match node that does just that.</p><p>For testing it out, you can use the <strong>Test</strong> feature just as before. But it is nice to actually run it via a browser (or a command-line using curl ) to be able to retrieve an actual authentication token for later use. So type this into your browser’s address bar:</p><pre><a href="https://bluebrokerat89xl.connect-platform.com/login?username=dude&amp;password=1234">https://&lt;project-codename&gt;.connect-platform.com/login?username=dude&amp;password=1234</a></pre><p>You can figure out &lt;project-codename&gt; by looking at your address bar when you are in your CONNECT instance’s panel, where you will see a URL like this:</p><pre><a href="https://bluebrokerat89xl.connect-platform.com/panel/editor?id=nt7mv5ep1wd">https://&lt;project-codename&gt;.connect-platform.com/panel/</a></pre><p>Where you will get a response like this:</p><pre>{&quot;token&quot;:&quot;eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJ1c2VybmFtZSI6ImR1ZGUiLCJpYXQiOjE1NDQyNzI2MDZ9.blv6egCUoJyyS-mWkWSC3LoH0DJnKe4RFjHk3iUQSxs&quot;}</pre><p>If you also want to see your /login/ node in action but want to test it by sending a request (using browser, curl , or any other proper tool), you can also press the <strong>Watch</strong> button, send your request, and come back and see your logic running. <strong>Watch</strong> works like <strong>Test</strong>, except instead of asking the inputs and running the node for recording, will instead wait for the first external request to the node and then record its execution flow:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/1*v4KDt94T1-fOzhq16F2RJQ.gif" /><figcaption><strong>fig. 17</strong>, watching a node execute</figcaption></figure><h3>STEP 5: Returning a User’s pet_name</h3><p>So the last remaining step is to add a node that receives an authentication token, verifies it, and returns the pet_name of the user the token belongs to:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*kLTuZvMORQ693jd72OJg4Q.png" /><figcaption><strong>fig. 18</strong>, consuming the authentication token</figcaption></figure><p>Now for testing it, copy the token from the previous step and then go to this URL:</p><pre><a href="https://bluebrokerat89xl.connect-platform.com/pet-name?token=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJ1c2VybmFtZSI6ImR1ZGUiLCJpYXQiOjE1NDQyNzI2MDZ9.blv6egCUoJyyS-mWkWSC3LoH0DJnKe4RFjHk3iUQSxs">https://&lt;project-codename&gt;.connect-platform.com/pet-name<br>?token=</a>&lt;token&gt;</pre><p>which will then yield a response like this:</p><pre>{&quot;pet_name&quot;:&quot;jojo&quot;}</pre><p>That’s it! We’ve successfully built a password-based authentication service that on the cloud, with a fully managed scalable database and backend (assuming you have used a CONNECT instance on the cloud)!</p><h3>What’s Next?</h3><p>First off, a friendly warning: I personally see passwords as an inherent security flaw on their own. People tend to use simple passwords and people tend to use similar passwords for different services, which means an authentication system based on password security will be vulnerable even if its own backend has remained completely secure. We used passwords for this tutorial because alternatives require more complicated setup (for example, for email based “magic link” authentication, you can easily use Mailjet combined with <strong>connect-mailjet</strong>, however you would need to register a valid domain name and also configure your domain name so that your emails don’t get discarded as spam). For production use-cases though, I would highly recommend always going for one of those alternatives since the slightly more complicated setup would not be a problem in that case.</p><p>What to do next? checkout <a href="https://medium.com/connect-platform/inline-coding-in-connect-platform-534fce3c8cdf">how to do inline Javascript coding on CONNECT platform</a>, or generally read the <a href="https://medium.com/connect-platform/guides/home">rest of the guides</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=402532d3486f" width="1" height="1" alt=""><hr><p><a href="https://medium.com/connect-platform/connect-102-an-authentication-service-402532d3486f">CONNECT 102: An Authentication Service</a> was originally published in <a href="https://medium.com/connect-platform">CONNECT platform</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[CONNECT 101: Hellow World!]]></title>
            <link>https://medium.com/connect-platform/connect-101-hellow-world-dee3e7c9720d?source=rss----94cd3f5eb256---4</link>
            <guid isPermaLink="false">https://medium.com/p/dee3e7c9720d</guid>
            <category><![CDATA[asynchronous]]></category>
            <category><![CDATA[backend-development]]></category>
            <category><![CDATA[connect-platform]]></category>
            <category><![CDATA[javascript]]></category>
            <category><![CDATA[guides-and-tutorials]]></category>
            <dc:creator><![CDATA[Eugene Ghanizadeh]]></dc:creator>
            <pubDate>Wed, 28 Nov 2018 14:54:07 GMT</pubDate>
            <atom:updated>2018-11-28T14:54:06.679Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*nqKUxtTQq_lRzui0NJWv_A.png" /></figure><p>In this article, we are going to use <a href="https://connect-platform.com">CONNECT</a> to do the famous Hellow World! program. To be more specific, we will create a micro-service on the cloud whose responsibility to say “Hellow World!” (or “Hellow” to other entities), and via doing so go over the basics of CONNECT platform.</p><h3>HELLOW WORLD!</h3><h4>STEP 0: Setup</h4><p>You need to setup an instance of CONNECT platform, and you can do that either on the Platform as a Service or on your local machine. If you have not done so already, head to <a href="https://deskfront.connect-platform.com/">https://deskfront.connect-platform.com/</a>, enter your email address, use the link in the email to log in, and click the “New Project” button, and you are good to go. You can check <a href="https://medium.com/connect-platform/how-to-setup-connect-platform-d82d49e029ee">this document</a> to see how can you set it up on your laptop as well.</p><h4>STEP 1: Create a Node</h4><p>Nodes are units of logic in CONNECT platform, and since responding with Hellow World! is also a bit of logic, we need to create a node. This is how you do that:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/1*bKzPP91ngbXDwwzScGRqgw.gif" /><figcaption><strong>fig. 1</strong>, creating a new node</figcaption></figure><p>Some observations:</p><ul><li>Look at the /some-path/ value which we changed to /hellow/ . Nodes in CONNECT are identified by a path (typically a URL), and so we changed it to /hellow/ since that is a better identifier of our logic.</li><li>Now look at the internal thingy on the top that we click and change into public . This indicates whether this node is to be accessible publicly or is it just for use internally in this project.</li></ul><h4>STEP 2: Add The Logic</h4><p>Now lets add the super-complex logic of responding with “Hellow World!”. The thinking is simple here, “Hellow World!” is a fixed value that we want to output. So:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/1*dw5IndBCExIvTngPG1j2Ew.gif" /><figcaption><strong>fig. 2</strong>, adding “Hellow World!” logic</figcaption></figure><p>Now lets test it! In the address bar of your browser, you’ll see something like</p><pre>https://&lt;projectcodename&gt;.connect-platform.com/panel/editor</pre><p>The https://&lt;projectcodename&gt;.connect-platform.com is the address of your micro-service. Remember that we set the path to /hellow/ and also change the node’s access from internal to public ? Try out this URL:</p><pre>https://&lt;projectcodename&gt;.connect-platform.com/hellow/</pre><p>and you will get something like this:</p><pre>{&quot;msg&quot;:&quot;Hellow World!&quot;}</pre><p>Observations:</p><ul><li>When you click on the big round + button, you’ll see four different types of stuff you can add, the first one being the value which we needed in our case. You can <a href="https://medium.com/connect-platform/connect-platform-basic-concepts-543b79c3094e">read this document</a> to understand more about them and other core concepts of CONNECT platform.</li><li>We also added an output, named it msg and connected (pun intended) the &quot;Hellow World!&quot; node to it. Thats why in the response object the key containing &quot;Hellow World!&quot; is named &quot;msg&quot; .</li></ul><h3>Hellow &lt;someone&gt;!</h3><p>Ok now lets change our node so that it can get a name and say &quot;Hellow&quot; to them, instead of just saying &quot;Hellow&quot; to &quot;World&quot; .</p><h4>STEP 0: Clean Up</h4><p>Delete the value containing &quot;Hellow World!&quot; . Click on it, and then click on the big round X button. Do not remove the output.</p><h4>STEP 1: Add an Input</h4><p>So we need to <em>take in</em> a name and say hellow to them, in other words, we need to add an “input”:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/1*_H71saq-y8DeZuoJP4p8QQ.gif" /><figcaption><strong>fig. 3</strong>, adding an input</figcaption></figure><h4>STEP 2: Add an Expression</h4><p>Now we need to combine the input name that we have on our node with &quot;Hellow&quot; , so a simple value will not do us any good. Instead we should use an expression:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/1*_flY9Z2MKBVOf1beN95ygw.gif" /><figcaption><strong>fig. 4</strong>, adding an expression</figcaption></figure><p>Note that our expression is supposed to combine &quot;Hellow&quot; with a given value. Thats why we added the “given value” or “input” on our expression as well, and named it “name”. Now lets describe what the expression should do:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/1*22_9_eQgoUL03IgyVbopYg.gif" /><figcaption><strong>fig. 5</strong></figcaption></figure><h4>STEP 3: Connect Everything</h4><p>Now, all we need to connect the input name of our node to input name of our expression, and connect the output of our expression to the output msg of our node:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/1*AWG0knE2apEHtSXqZg6JnQ.gif" /><figcaption><strong>fig. 6</strong></figcaption></figure><p>Now, if we simply try the URL we tried last time, we will get this error:</p><pre>input name is missing from {}</pre><p>Which indicates that input name is missing. To pass it the input as well, we can try this URL instead:</p><pre>https://&lt;projectcodename&gt;.connect-platform.com/hellow/?name=John</pre><p>which will result in:</p><pre>{&quot;msg&quot;:&quot;Hellow John&quot;}</pre><h3>What’s Next?</h3><p>Now that you have caught up with some of the basics of CONNECT platform, it is a good idea to <a href="https://medium.com/connect-platform/connect-platform-basic-concepts-543b79c3094e">read up on its core concepts</a>, or you can take a look at the <a href="https://medium.com/connect-platform/guides/home">rest of the guides</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=dee3e7c9720d" width="1" height="1" alt=""><hr><p><a href="https://medium.com/connect-platform/connect-101-hellow-world-dee3e7c9720d">CONNECT 101: Hellow World!</a> was originally published in <a href="https://medium.com/connect-platform">CONNECT platform</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Inline Coding in CONNECT platform]]></title>
            <link>https://medium.com/connect-platform/inline-coding-in-connect-platform-534fce3c8cdf?source=rss----94cd3f5eb256---4</link>
            <guid isPermaLink="false">https://medium.com/p/534fce3c8cdf</guid>
            <category><![CDATA[backend-development]]></category>
            <category><![CDATA[connect]]></category>
            <category><![CDATA[javascript]]></category>
            <category><![CDATA[asynchronous]]></category>
            <category><![CDATA[guides-and-tutorials]]></category>
            <dc:creator><![CDATA[Eugene Ghanizadeh]]></dc:creator>
            <pubDate>Thu, 22 Nov 2018 17:01:04 GMT</pubDate>
            <atom:updated>2018-11-22T17:01:04.435Z</atom:updated>
            <content:encoded><![CDATA[<p>As we discussed <a href="https://medium.com/connect-platform/connect-platform-basic-concepts-543b79c3094e">here</a>, the core idea of CONNECT platform is that main logical flow of backend code is asynchronous. However, not all levels of the logic are necessarily asynchronous, so you would need to easily mix the asynchronous graphs of CONNECT with traditional synchronous code.</p><p>Fortunately, CONNECT platform is based on NodeJS, and you can easily integrate Javascript code pieces into the platform. There are two main ways to do that:</p><ol><li>Inline Codes: <strong>value</strong> and <strong>expression</strong> nodes already parse Javascript expressions. You can even write functions with zero or one arguments and write arbitrary amount of code easily right in the middle of a graph.</li><li>Packages: for much more elaborate codes, like submodules, you can easily create Javascript modules, add a little bit of extra metadata that turns your module into a CONNECT package, and push it to some remote git repository. Then you can install it as a package on the “<em>packages</em>” section of your CONNECT project easily. You can even publish the packages to NPM and allow fellow CONNECT users to utilize your modules.</li></ol><p>You can read on how to create CONNECT packages here. However, here we are going to cover how can you do inline code pieces on the platform.</p><h4>Inline Codes Using Expressions</h4><p>So this is how you would create an expression that gets a number and multiplies it by two:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/1*l0wkyvgJF2gbqb5EY334pg.gif" /><figcaption><strong>fig. 1</strong>, how to add a usual expression</figcaption></figure><p>Now imagine if you wanted to get a list of numbers and multiply it by two, and for some reason you did not want to use Javascript’s filter, then you would do something like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/1*LIayGA2IvrGj4tkhQ_WjvA.gif" /><figcaption><strong>fig. 2</strong>, doing inline code in an expression</figcaption></figure><p>Basically, when the object described in an <strong>expression</strong> or <strong>value</strong> is a function with zero arguments, the node will call the function and output its return value. In the context of this function, you also have access to all the inputs as they are named, as you can see in figure 2, that you can easily access a in the function’s body.</p><p>In the function’s context, you also have access to these other objects:</p><ul><li>console which you can use for doing console logs and what not.</li><li>require function, which you can use to require any module you might need.</li><li>error function, which you can utilize to indicate an error has occurred. You can either pass it an error message (a string) or an Error object.</li><li>context : context is a shared object that is created for each request. All nodes that are invoked while processing that request can have access to it, read from it and write to it. For example, you can use context.req to access the express request object.</li></ul><p>If you want to do a little bit of asynchronous logic in your code as well, then you can uses a function with one argument instead of zero:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/9a5c17acf4f7ea86d6bc03b999e944a8/href">https://medium.com/media/9a5c17acf4f7ea86d6bc03b999e944a8/href</a></iframe><p>For example, this code in an expression will run a node on path /something on items of a list. The done object is a function that you can call when you are done, and the data you pass to it would be the output of the expression. You can also call it without any arguments, and the output of the expression would be undefined .</p><p>Fun note: the platform.call is actually the way you would invoke a node from within Javascript, regardless of it being native itself (written in Javascript) or a graph created visually.</p><h3>Whats Next?</h3><p>Check out the <a href="https://medium.com/connect-platform/guides/home">rest of the guides</a> maybe? ;)</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=534fce3c8cdf" width="1" height="1" alt=""><hr><p><a href="https://medium.com/connect-platform/inline-coding-in-connect-platform-534fce3c8cdf">Inline Coding in CONNECT platform</a> was originally published in <a href="https://medium.com/connect-platform">CONNECT platform</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How To Setup CONNECT platform]]></title>
            <link>https://medium.com/connect-platform/how-to-setup-connect-platform-d82d49e029ee?source=rss----94cd3f5eb256---4</link>
            <guid isPermaLink="false">https://medium.com/p/d82d49e029ee</guid>
            <category><![CDATA[guides-and-tutorials]]></category>
            <category><![CDATA[connect-platform]]></category>
            <category><![CDATA[visual-programming]]></category>
            <category><![CDATA[nodejs]]></category>
            <category><![CDATA[low-code]]></category>
            <dc:creator><![CDATA[Eugene Ghanizadeh]]></dc:creator>
            <pubDate>Mon, 01 Oct 2018 20:43:36 GMT</pubDate>
            <atom:updated>2018-12-08T13:16:59.385Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*4Ir7XyKawSHvBZW0tTByQg.png" /></figure><p>The easiest way to setup instances of CONNECT platform is to use the <strong>platform as a service</strong>, that will take care of everything and setup CONNECT instances for you with a few clicks.</p><h3>#0 Using The Platform As a Service</h3><p>1 Go to <a href="https://deskfront.connect-platform.com/auth">this link</a>. Enter your email address. You will receive an email with a link in it, open the link (click on it or copy it in your browser).</p><p>2 Click on the New Project button. Enter the name of your project. Now you have a working instance on the cloud equipped with CONNECT platform, so start building your projects ;)</p><p>However, you might also want to setup local instances to work with the platform locally, and this guide aims to help you with that.</p><p>There are multiple ways to setup CONNECT platform locally, based on tools you have already installed and technologies you are comfortable with.</p><h3>#1 Using The Docker Image</h3><p>Well if you are the docker kind of guy, we are as well, so we have <a href="https://hub.docker.com/r/loreanvictor/connect-platform/">a super nice docker image</a> for you. (If you are not the docker kind of guy, this is as good of an excuse to become one as any, just get it <a href="https://www.docker.com/get-started">here</a> and then keep reading on).</p><p>1 Create a folder for your project. needless to say, open a terminal and change directory to that folder.</p><p>2 Run the following command. This command will simply run the image, bind to the port 4000 , and volume folders from your directory so that your data is also saved (so yes you can also change the parameters as you wish).</p><pre>docker run -dit \<br>          --name connect-platform \<br>          -p 4000:4000 \<br>          -v $(pwd)/panel-generated:/app/panel-generated \<br>          -v $(pwd)/secure:/app/secure \<br>          --env CONNECT_INSTALL_EXTERNAL_PACKAGES=true \<br>          loreanvictor/connect-platform</pre><p>check it out on localhost:4000/panel .</p><h3><strong>#2 Using The Starter Repository</strong></h3><p>There is a <a href="https://github.com/CONNECT-platform/connect-platform-boilerplate">starter github project</a> to ease up setting up a new instance locally. It is extremely easy to use. For it to work, you need to have <a href="https://git-scm.com">git</a>, <a href="https://nodejs.org/en/">nodejs</a> and <a href="https://www.npmjs.com/get-npm">npm</a> installed. This is how it works:</p><p>1 Create a folder for your project, open a terminal and change the directory of your terminal to that folder.</p><p>2 Clone the starter project:<br>git clone https://github.com/loreanvictor/connect-platform-biolerplate.git .</p><p>3 Use NPM to install the dependencies of the project:<br>npm install</p><p>4 Start your instance:<br>npm start</p><p>check it out at localhost:4000/panel .</p><h3>#3 Do Everything On Your Own</h3><p>If you for some reason want to take control of everything, you are more than welcome to. At its core, CONNECT platform is a simple NPM package that can be installed on any other NPM project. CONNECT utilizes NPM to manage its own packages as well, and it even utilizes ExpressJS to run its web-server, so you can even pass it an independent instance of ExpressJS instead of having it setup its own from scratch (if, for example, you want to run a CONNECT instance in parallel to an existing ExpressJS app).</p><p>0 Make sure you have NodeJS and NPM installed, and you have setup your NPM project already.</p><p>1 Install CONNECT platform on your project:<br> npm install connect-platform --save</p><p>2 Also using <a href="https://nodemon.io">Nodemon</a> to run the instance is highly recommended, as CONNECT would change the data and rely on you to restart the instance for those changes to take effect: npm install nodemon --save-dev (of course you can use your own favorite code watcher Node restarter tool as well).</p><p>3 Configure your CONNECT instance. This gist should give you the general idea of how to do that:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/3bcd83a5a425f51f8f3058c90eb6442d/href">https://medium.com/media/3bcd83a5a425f51f8f3058c90eb6442d/href</a></iframe><p>If you have your own ExpressJS app instance, you can simply pass it to CONNECT so it would use that one instead of creating its own. Just add this line before the platform.start() :</p><pre>// assuming the name of your express instance<br>// is app.<br>platform.init(app);</pre><p>The platform.start() line actually loads all modules required by the CONNECT instance, creates an Express app if it have not been passed one yet and binds itself to it. You can take control of all of that process and start the server on your own terms. Just do something like this instead of platform.start() :</p><pre>platform.init(app).load().bind();</pre><p>and then start your app however you desire.</p><p>4 Run the whole thing:<br> nodemon . (of course if you did not choose to use Nodemon this command will change accordingly).</p><p>check it out on localhost:4000/panel .</p><h3># Whats Next?</h3><p>Well, read <a href="https://medium.com/connect-platform/guides/home">the rest of the guides</a>, or start with this <a href="https://medium.com/connect-platform/connect-101-hellow-world-dee3e7c9720d?source=collection_category---5------1---------------------">“Hellow World!” tutorial</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d82d49e029ee" width="1" height="1" alt=""><hr><p><a href="https://medium.com/connect-platform/how-to-setup-connect-platform-d82d49e029ee">How To Setup CONNECT platform</a> was originally published in <a href="https://medium.com/connect-platform">CONNECT platform</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[CONNECT platform: Core Concepts]]></title>
            <link>https://medium.com/connect-platform/connect-platform-basic-concepts-543b79c3094e?source=rss----94cd3f5eb256---4</link>
            <guid isPermaLink="false">https://medium.com/p/543b79c3094e</guid>
            <category><![CDATA[connect-platform]]></category>
            <category><![CDATA[software-development]]></category>
            <category><![CDATA[javascript]]></category>
            <category><![CDATA[node-js-tutorial]]></category>
            <category><![CDATA[low-code]]></category>
            <dc:creator><![CDATA[Eugene Ghanizadeh]]></dc:creator>
            <pubDate>Sun, 15 Apr 2018 21:46:16 GMT</pubDate>
            <atom:updated>2018-12-08T13:16:11.516Z</atom:updated>
            <content:encoded><![CDATA[<p>So in this document, we’ll go over some of the core concepts of CONNECT platform.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*nqKUxtTQq_lRzui0NJWv_A.png" /></figure><h3>Core Idea</h3><p>A text-based code looks generally like this:</p><pre>doA();<br>doB();<br>doC();<br>doD();<br>doE();</pre><p>which can then be simply read as “<em>do A, then do B, then do C,</em>” etc. If we were to show the “<em>then</em>” word in this description on the code, for example using arrows, it would look like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*KNoyuMY1TK8r3ClYX1oOLQ.png" /><figcaption><strong>fig. 1</strong>, execution flow of synchronous code</figcaption></figure><p>However, backend logic is generally asynchronous in nature, i.e. it is something like “<em>do A and B, when A is done then do C, when B is done then do D, and when A, B and C are done then do E</em>”, which would look something like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*07QToB-gNjNhoUwGmpUCbw.png" /><figcaption><strong>fig. 2</strong>, execution flow of asynchronous code</figcaption></figure><p>If this looks confusing to you, it is because it indeed is. This is an abstract schematic of a <em>callback hell</em>, and stems from the fact that the flow of asynchronous logic is not a <strong>sequence</strong>, but rather a full on <strong>graph</strong>.Our callback hell is simply a result of us trying to represent a <strong>graph</strong> in a <strong>sequential</strong> format, i.e. we could simply solve the problem at hand by representing the logical flow like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Bvh714vczFzC0oPeNA4C9g.png" /><figcaption><strong>fig. 3</strong>, execution flow of asynchronous code, represented by a graph instead of a sequence</figcaption></figure><p>This, is the core idea of CONNECT platform, that backend logic is asynchronous in nature and so should be represented in a graph-based (i.e. graphical) format so that the “code” (so to speak) would be reflective of what is actually going to happen:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*4nafhYHYc85-4mVSyk8ypQ.png" /><figcaption><strong>fig. 4</strong>, a sample node on CONNECT platform</figcaption></figure><h3>Nodes</h3><p>The same way that sequential codes are made of <strong>statements</strong>, codes on CONNECT platform are made of <strong>nodes</strong>. In the screenshot above, every box that you see in the main area is a <strong>node</strong>. In figure 1 and 2, each box represents a <strong>statement</strong>, and boxes in figure 2 correspond to boxes on figure 3, each of them being a <strong>node</strong>.</p><p>To see how the graph shows execution, lets take another look at figure 3:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Bvh714vczFzC0oPeNA4C9g.png" /><figcaption><strong>fig. 3</strong>, brought up here again so that you would not need to scroll up constantly.</figcaption></figure><p>Again, this graphs means “<em>do A and B, when A is then done do C, when B is done then do D, and when A, B and C are then done do E</em>”. The rule is simple, <strong>links </strong>are <em>activated </em>when the <strong>node</strong> they start from is done, and each <strong>node</strong> gets executed when all the <strong>links</strong> pointing to it are <em>activated </em>(or there are no <strong>links</strong> to them). So in the beginning, doA() and doB() start since there are no links to them. doC() is executed exactly when doA() is done, similarly doB() will start when doB() is finished, and doE() needs to wait for doA() , doB() and doC() to start working.</p><p><strong>Nodes</strong> can have multiple inputs and multiple possible outputs. Data is transferred between the <strong>nodes</strong> via the <strong>links</strong> between the <strong>nodes</strong>, which means a <strong>node</strong> will basically wait for all of its input to be provided before starting execution. So for example, the the example below, the node /authenticate will start only when the token parameter is provided to it.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*5oTFPJ0iGCaKFWD-f882Ng.png" /><figcaption><strong>fig. 5</strong>, a sample node</figcaption></figure><p>The <strong>node</strong> also has two possible outputs: either a user is returned (and passed via the link to the next <strong>node</strong> waiting), or the token is not correct, which causes the bad_token link to be activated. Note that the execution of each <strong>node</strong> <em>can only result in one of its possible outputs</em>, so for example the /authenticate node cannot output both user and bad_token .</p><p>Another important thing to notice is that bad_token is in a different color than user . That is because bad_token is simply a different kind of output, a <strong>control output</strong>. It is called a <strong>control output</strong> because it does not give out any data. It merely signals that the given token was not correct. As you can see, the link going out of bad_token is also of a different color, again to indicate that this link does not carry over data, but rather represents a <strong>control state</strong>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*UWC1YcWZD3i72n6RRGYs3A.png" /><figcaption><strong>fig. 6</strong>, a node that awaits both some data input and some control state</figcaption></figure><p>Another example can be seen in the figure above. The /user/subscribed node is supposed to take in a user and see if they are subscribed or not, respectively resulting in not_subscribed and subscribed control outputs. The other node in the figure (which is an <strong>expression node</strong> getting an article and outputting the same article ) has one input ( article ) but also there is a <strong>control link</strong> attached to it from the /user/subscribed node’s subscribed control output. This simply means that this node will not start when its input is ready and provided, but it also waits for the <strong>control links</strong> connected to it to all activate as well, or to put it in another manner, the <strong>control links</strong> basically just control the execution of other <strong>nodes</strong> without transference of data.</p><p>Note that if multiple normal links (green ones) are connected to a specific input of a node, the node will not wait for all of them to start, as just one of them providing the input would be enough. On the other hand, a node will wait for all of the control links connected to it to activate before starting.</p><h3>Node Signatures</h3><p>On figure 4, you can see that the whole graph also has two inputs article_id and user_token and three possible outputs, article , not_found and not_allowed the latter two being <strong>control outputs</strong>. Links will carry data from these inputs to the nodes on the graph, normal outputs can transfer their data to the article output, and control outputs can be connected to the control outputs of the whole piece. The whole graph is a <strong>node</strong> on its own.</p><p>Besides the inputs and the outputs, each node has other properties, all of which you can see on the top part of the left-bar on the panel:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/400/1*EoocP4mkSD2ZL48O9RsSog.png" /><figcaption><strong>fig. 7</strong>, a node’s signature</figcaption></figure><ul><li><strong><em>internal</em></strong><em><br></em> This means this node is only to be used by other nodes on your project. It could alternatively be <strong>public</strong>, which would mean it would be accessible by requests to the url of your instance from outside.</li><li><strong><em>GET</em><br></strong> This is the “method” of the node and can be either <strong>GET</strong>, <strong>POST</strong>, <strong>PUT</strong> or <strong>DELETE</strong>. For public nodes, this indicates the HTTP method of incoming requests. Even for internal nodes, it is a good idea to use the method to indicate the type of operation that the node would do.</li><li>/some-path/ <br>This is the “path” of the node, indicating how the node is to be accessed, either by other nodes or in case of public nodes by a request to some URL (and so it must have a standard URL format). So for example, if this node was made public, you could access it by requests to<br> &lt;project-codename&gt;.connect-platform.com/some-path/ .<br>Note that the “path” is the <em>only identifier</em> for each node, i.e. for example you cannot have two nodes with the same “path” but different “method”s.</li></ul><p>All of the meta-data of a node, including its access-level (<strong>internal</strong> vs <strong>public</strong>), its method, its path, alongside its inputs, outputs and control outputs are called the node’s <strong>signature</strong>. You can checkout the list of all of the signatures of the public nodes on your project on &lt;project-codename&gt;.connect-platform.com/api .</p><p><strong>NOTE: </strong>the method of public nodes also affects how you can pass the required inputs to them. CONNECT will automatically search in query parameters, request header, request body and even the URL itself (in case of parametric URLs) to extract the inputs. However, it is customary to pass the data on the request based on the method being invoked, so on GET and DELETE requests all inputs should be passed via the query parameters and on POST and PUT requests you can also utilize the request body to pass inputs.</p><h3>Node Types</h3><p>If you want to add nodes to your code, you need to click on the big circular + button on the top of the right-side bar. When you do this, a list of possible node types appears:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/600/1*RqbWIpzJjCEkrc3seu1WyA.gif" /><figcaption><strong>fig. 8</strong>, various node types</figcaption></figure><h4>Values and Expressions</h4><p>Values and Expressions are nodes that will evaluate their given Javascript expression based on their given input and output the result:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/618/1*QOi4wN7ctN8YVzWEuM82EA.png" /><figcaption><strong>fig. 9</strong>, an expression node</figcaption></figure><p>The expression can be any Javascript expression, and the inputs of the expression are injected into its context. You can actually easily do sizable amounts of Javascript code inside the <strong>expression node</strong>, which is important for effortlessly mixing asynchronous graphs with pieces of synchronous code. Just click on the icon on the top-right of the box, and you will get a complete Javascript editor to write your code.</p><p>The same holds for <strong>values</strong>. In fact, <strong>values</strong> are simply <strong>expressions</strong> that do not have any inputs.</p><h4>Calls</h4><p>Calls invoke other nodes with given paths. So for example, figure 10 contains two call nodes, one that invokes another node at path /authenticate (supposedly for authenticating the user using the given token ), and another one at path /user/subscribed (which again, supposedly will check if the user is subscribed or not.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*AGskfFTzNQIUOb_RKUW2yw.png" /><figcaption><strong>fig. 10</strong>, call nodes</figcaption></figure><p>You can use call nodes to invoke both <strong>internal</strong> and <strong>public</strong> nodes. In fact, <strong>internal</strong> nodes can only be invoked using <strong>calls</strong>.</p><p>A node might also contain a call to its own path, in which case it would be similar to a function recursively calling itself.</p><h4>Switches</h4><p>Switches act as conditional controllers for CONNECT platform. They are equivalent to switch-case in Javascript for example. A <strong>switch </strong>has<strong> </strong>one input and multiple <strong>control outputs</strong>, which will be activated based on the value of the input. So for example, in figure 11 an expression calculates n &gt; 2 based on its given input, and its output is passed to a <strong>switch</strong> with two <strong>control outputs</strong>, one for the value true (so it will be activated when n indeed is greater than 2 ) and another for the value false (so it will be activated otherwise).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*UL5sdW3mHSkA71x0wCSJkg.png" /><figcaption><strong>fig. 11</strong>, a switch node</figcaption></figure><p>Each <strong>control output</strong> on a switch can also be mapped to <em>any </em>Javascript expression, and the first whose value matches the given input will be activated. You can also write ... for the expression of a <strong>control output</strong> of a <strong>switch</strong>, which is translated as <em>any value </em>(or is equal to a switch-case ‘s default option).</p><h3>Whats Next?</h3><p>Checkout the <a href="https://medium.com/connect-platform/guides/home">rest of the guides</a>, or <a href="https://medium.com/connect-platform/connect-102-an-authentication-service-402532d3486f">checkout a more pragmatic tutorial on building an authentication service with CONNECT</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=543b79c3094e" width="1" height="1" alt=""><hr><p><a href="https://medium.com/connect-platform/connect-platform-basic-concepts-543b79c3094e">CONNECT platform: Core Concepts</a> was originally published in <a href="https://medium.com/connect-platform">CONNECT platform</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>