<?xml version="1.0" encoding="UTF-8"?> <feed xmlns="http://www.w3.org/2005/Atom"><link href="https://scurker.com"/><title>scurker.com</title><updated>2018-12-12T00:00:00+00:00</updated><id>https://scurker.com/</id><author><name>Jason Wilson</name></author><atom:link xmlns:atom="http://www.w3.org/2005/Atom" rel="self" type="application/atom+xml" href="https://scurker.com/feed"/><!-- feed entries --><entry><title>Primer to Measuring Page Speed</title><link href="https://scurker.com/primer-to-measuring-page-speed"/><id>https://scurker.com/primer-to-measuring-page-speed</id><updated>2018-12-12T00:00:00+00:00</updated><content type="html"><p>That are many different things that contribute to the speed of a page load. The amount of javascript that is being loading, the code that is being run in javascript, or even how assets are being loaded. Understanding the things that impact a page load can be overwhelming, but making small improvements can make a gradual impact towards making pages load faster.</p> <p>But how do you know if your site is slow or fast?</p> <blockquote class="twitter-tweet" align="center" data-dnt="true"><p lang="en" dir="ltr">Want to make faster websites?<br><br>Measure it.<br><br>We implemented something that is supposed to be faster &#x2014; but it&#x2019;s not, and we only knew because we measured it.</p>&#x2014; Jason (@scurker) <a href="https://twitter.com/scurker/status/1057719401292673024?ref_src=twsrc%5Etfw">October 31, 2018</a></blockquote> <p>Measuring is a key component of helping to make sites faster. Without having some baseline measurements to compare changes against, or knowing what performance targets you want to hit - it&apos;s very difficult to actually know if the changes being made are having a positive effect.</p> <h2 id="key-terms"><a class="anchor-link" aria-hidden="true" href="#key-terms"></a>Key Terms</h2> <p>In order to first measure something, one must understand the different terms of measurement. A page load isn&apos;t made up of a single specific metric as there are different aspects of loading that happen over a span of time. Understanding these measurements help point out specific areas to profile and improve.</p> <p>However, what we particularly care about are the measurements that are directly related to how a user <em>perceives</em> how fast a given page is.</p> <h3 id="time-to-first-byte-ttfb"><a class="anchor-link" aria-hidden="true" href="#time-to-first-byte-ttfb"></a>Time to First Byte (TTFB)</h3> <p><abbr title="Time to first byte">TTFB</abbr> is a key metric for measuring server performance. This metric measures the time from the initial client request, to when the client receives its first byte. It&apos;s important to note the user will never see content during this phase.</p> <h3 id="first-contentful-paint-fcp"><a class="anchor-link" aria-hidden="true" href="#first-contentful-paint-fcp"></a>First Contentful Paint (FCP)</h3> <p>After the initial request starts to return content, <abbr title="First contentful paint">FCP</abbr> helps to measure when the user first sees some <em>meaningful</em> content, such as text or images. This is the first indication to the user that something is happening.</p> <h3 id="first-meaningful-paint-fmp"><a class="anchor-link" aria-hidden="true" href="#first-meaningful-paint-fmp"></a>First Meaningful Paint (FMP)</h3> <p><abbr title="First meaningful paint">FMP</abbr> this is the first paint activity that is actually meaningful and contains content that is useful to the user. There&apos;s no standard definition of when this happens, as the content that is meaningful to the user can very from page to page. In order to track <abbr title="First meaningful paint">FMP</abbr>, you will need to place specific user timings to track what you would consider to be the first meaningful paint to be, such as above the fold content or specific hero elements.</p> <h3 id="time-to-interactive-tti"><a class="anchor-link" aria-hidden="true" href="#time-to-interactive-tti"></a>Time to Interactive (TTI)</h3> <p><abbr title="Time to interactive">TTI</abbr> happens after the first bit of content renders, and when the page responds to user interactions within 50ms. Long gaps between <abbr title="First contentful paint">FCP</abbr> or <abbr title="First meaningful paint">FMP</abbr> and <abbr title="Time to interactive">TTI</abbr> can lead to a bit of an &quot;uncanny valley&quot; experience for users, since a page may appear to be ready but user interactions are slow or unresponsive.</p> <h3 id="dom-content-loaded"><a class="anchor-link" aria-hidden="true" href="#dom-content-loaded"></a>DOM Content Loaded</h3> <p><code>DOMContentLoaded</code> is an event when the DOM is ready and there are no more stylesheets blocking javascript execution. This may not be a significant metric for users since it may not actually mirror the experience of when it appears to a user that a page is loaded.</p> <h3 id="load"><a class="anchor-link" aria-hidden="true" href="#load"></a>Load</h3> <p>Finally, the <code>load</code> event that happens after everything on a page has completely loaded. However, a user may be able to interact and view meaningful content well before this event occurs.</p> <p>Here&apos;s a simplified visualization of what these events might look like on a timeline:</p> <img src="/assets/images/posts/timeline-performance-metrics.svg" alt="timeline performance metrics" width="800"> <p>Of these events, the ones we care about are the ones that help answer how a user might perceive the given speed of a page:</p> <ul> <li><em>Is something happening?</em> - First Contentful Paint (<abbr title="First contentful paint">FCP</abbr>)</li> <li><em>Is it useful?</em> - First Meaningful Paint (<abbr title="First meaningful paint">FMP</abbr>)</li> <li><em>Can it be used?</em> - Time to Interactive (<abbr title="Time to interactive">TTI</abbr>)</li> </ul> <p>Now that we know a little bit more about some of the different page speed metrics, let&apos;s take a look at some sample of tools that help to profile and provide insights into your page speed.</p> <h2 id="lighthouse"><a class="anchor-link" aria-hidden="true" href="#lighthouse"></a>Lighthouse</h2> <p><a href="https://developers.google.com/web/tools/lighthouse/">Lighthouse</a> is built into Google Chrome&apos;s developer tools along with being available as a <a href="https://github.com/GoogleChrome/lighthouse">CLI or via a node module</a>. Lighthouse is a great place to start to capture a quick overview of common performance metrics as well as scoring a lot of other useful things.</p> <p><img src="/assets/images/posts/lighthouse-audit-results.png" alt="scurker.com lighthouse audit results"></p> <p>Using Lighthouse directly from Chrome&apos;s devtools is the easiest way to get started, and provides the 3 key metrics we&apos;re looking for: <abbr title="First contentful paint">FCP</abbr>, <abbr title="First meaningful paint">FMP</abbr>, and <abbr title="Time to interactive">TTI</abbr>. From the report, you can view tips and suggestions on things to improve to help make your page faster along with viewing the trace to dig into the details.</p> <p>Since you are likely running Lighthouse locally, the results can vary from what real users may experience. Results can be skewed by other Chrome extensions, background programs, or even network behavior. For the cleanest results it&apos;s recommended to run Lighthouse in either an incognito window or in a separate Chrome profile without any Chrome extensions enabled.</p> <p>As previously mentioned, Lighthouse is a great place to start - but may not accurate reflect what your real-world users are experiencing. Which brings us to our next tool...</p> <h2 id="webpagetestorg"><a class="anchor-link" aria-hidden="true" href="#webpagetestorg"></a>Webpagetest.org</h2> <p><a href="https://www.webpagetest.org/">Webpagetest.org</a> is a great tool for measuring speed from real world locations with real user connection speeds on a variety of different browsers and devices.</p> <p><img src="/assets/images/posts/web-page-test.png" alt="web page test site"></p> <p>Since WebPageTest runs on real devices there&apos;s a limited pool available so test times may vary depending on the number of people running tests.</p> <p>Given the variety of options, it&apos;s easy to configure the different options to get a ballpark metric of what real world users are seeing:</p> <ul> <li>Connection Speed</li> <li>Number of Tests to Run (more tests will provide an average of the results and be less susceptible to abnormal spikes)</li> <li>Repeat or First View (comparing between initial (uncached) or repeat (cached) times)</li> <li>Capture video (provide a film strip of the page load over time)</li> </ul> <p><img src="/assets/images/posts/web-page-test-results.png" alt="web page test scurker.com results"></p> <p>The output from WebPageTest provides a cornucopia of results and metrics, along with a resource of other items to review. The 2 of the 3 key metrics are provided as well, Start Render (<abbr title="First contentful paint">FCP</abbr>) and First Interactive (<abbr title="Time to interactive">TTI</abbr>). In the results you can review your page&apos;s waterfall, which mirrors what you would see in Chrome&apos;s dev tools.</p> <p>For more advanced metrics (such as <abbr title="First meaningful paint">FMP</abbr>) there is a <a href="https://sites.google.com/a/webpagetest.org/docs/using-webpagetest/scripting">custom metrics option</a> that allows you to run arbitrary javascript at the end of a test to collect custom metrics. From here you could retrieve a specific <a href="https://developer.mozilla.org/en-US/docs/Web/API/User_Timing_API">user timing</a> to be returned to the WebPageTest front-end.</p> <p>There are limitations to the public version of WebPageTest, as you are sharing instances with other users but there is a <a href="https://github.com/WPO-Foundation/webpagetest-docs/blob/master/user/Private%20Instances/README.md">self hosting option</a> if you need more custimization or reliability.</p> <h2 id="puppeteer"><a class="anchor-link" aria-hidden="true" href="#puppeteer"></a>Puppeteer</h2> <p><a href="https://github.com/GoogleChrome/puppeteer">Puppeteer</a> is a node library that provides a high-level API to control Chrome/Chromium and allows for the ultimate flexibility in capturing page speed metrics. While Puppeteer may not be as simple to use as the previous tools, it does offer an extended ability to track additional custom metrics.</p> <h3 id="getting-started-with-puppeteer"><a class="anchor-link" aria-hidden="true" href="#getting-started-with-puppeteer"></a>Getting Started with Puppeteer</h3> <p>To get started with Puppeteer, you either need to have it installed globally (<code>npm install -g puppeteer</code>) or locally inside of a project (<code>npm install puppeteer</code>).</p> <p>With Puppeteer you have access to the <a href="https://github.com/GoogleChrome/puppeteer/blob/master/docs/api.md#class-browser"><code>browser</code></a> to control the Chrome instance, and <a href="https://github.com/GoogleChrome/puppeteer/blob/master/docs/api.md#class-page"><code>page</code></a> allowing you to control various page aspects such as the viewport size, or navigation. Additionally creating a CDPSession is used to access the <a href="https://github.com/GoogleChrome/puppeteer/blob/master/docs/api.md#targetcreatecdpsession">Chrome Devtools Protocol</a> for things that are not natively available through the Puppeteer api.</p> <blockquote> <p>These examples are heavily dependant upon <a href="https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Statements/async_function"><code>async</code></a> and <a href="https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/await"><code>await</code></a> so for native Node support, you will need to be using Node 8 or greater.</p> </blockquote> <pre><code class="language-javascript"><span class="keyword">const</span> puppeteer = <span class="built_in">require</span>(<span class="string">&apos;puppeteer&apos;</span>); (<span class="keyword">async</span> <span class="function"><span class="keyword">function</span>(<span class="params"></span>) </span>{ <span class="keyword">const</span> browser = <span class="keyword">await</span> puppeteer.launch(); <span class="keyword">const</span> page = <span class="keyword">await</span> browser.newPage(); <span class="keyword">const</span> client = <span class="keyword">await</span> page.target().createCDPSession(); <span class="comment">// capture results</span> })();</code></pre> <h3 id="navigating-to-a-page"><a class="anchor-link" aria-hidden="true" href="#navigating-to-a-page"></a>Navigating to a Page</h3> <pre><code class="language-javascript"><span class="keyword">await</span> page.goto(<span class="string">&apos;https://example.com&apos;</span>);</code></pre> <span class="demo-link"> <a href="https://github.com/scurker/page-speed-metrics-with-puppeteer/blob/master/examples/navigate-to-page.js">view navigating to a page example</a> </span> <p>The above step works for pages where no authentication or user interaction is required. For more complex steps, Puppeteer allows for complete control of the page to perform any interactions that typically require a user.</p> <pre><code class="language-javascript"><span class="keyword">await</span> page.goto(<span class="string">&apos;https://example.com&apos;</span>); <span class="keyword">await</span> page.type(<span class="string">&apos;#username&apos;</span>, <span class="string">&apos;username&apos;</span>); <span class="keyword">await</span> page.type(<span class="string">&apos;#password&apos;</span>, <span class="string">&apos;password&apos;</span>); <span class="keyword">await</span> page.$<span class="built_in">eval</span>(<span class="string">&apos;#login&apos;</span>, form =&gt; form.submit()); <span class="keyword">await</span> page.waitForNavigation();</code></pre> <span class="demo-link"> <a href="https://github.com/scurker/page-speed-metrics-with-puppeteer/blob/master/examples/navigate-to-page-with-login.js">view authentication example</a> </span> <h3 id="simulating-network-speed"><a class="anchor-link" aria-hidden="true" href="#simulating-network-speed"></a>Simulating Network Speed</h3> <p>For more &quot;real world&quot; page speed metrics, we need to simulate slower network conditions. Puppeteer can emulate these conditions using the <a href="https://github.com/GoogleChrome/puppeteer/blob/master/docs/api.md#targetcreatecdpsession">Chrome Devtools Protocol</a> and <a href="https://chromedevtools.github.io/devtools-protocol/tot/Network#method-emulateNetworkConditions"><code>Network.emulateNetworkConditions</code></a>. <code>Network.emulateNetworkConditions</code> expects throughput in bits vs bytes, so you will need to divide any byte value by <code>8</code> for accurate results.</p> <pre><code class="language-javascript"><span class="comment">// Simulate 3G Conditions</span> <span class="keyword">await</span> client.send(<span class="string">&apos;Network.emulateNetworkConditions&apos;</span>, { <span class="attr">offline</span>: <span class="literal">false</span>, <span class="attr">downloadThroughput</span>: <span class="number">1.6</span> * <span class="number">1024</span> * <span class="number">1024</span> / <span class="number">8</span>, <span class="comment">// 1.6 mb/s</span> uploadThroughput: <span class="number">768</span> * <span class="number">1024</span> / <span class="number">8</span>, <span class="comment">// 400 kb/s</span> latency: <span class="number">300</span> <span class="comment">// 300ms</span> });</code></pre> <span class="demo-link"> <a href="https://github.com/scurker/page-speed-metrics-with-puppeteer/blob/master/examples/simulate-network-speed.js">view simulate network speed example</a> </span> <h3 id="tracking-cached-vs-first-view"><a class="anchor-link" aria-hidden="true" href="#tracking-cached-vs-first-view"></a>Tracking Cached vs First View</h3> <p>For measuring a first view request, there&apos;s two things that need to happen: prevent requests from being fetched from the cache and preventing requests from being fetched via a service worker (if available). There&apos;s no direct access (<a href="https://github.com/GoogleChrome/puppeteer/issues/2634">yet</a>) to shutting down a service worker, but service workers can be <a href="https://chromedevtools.github.io/devtools-protocol/tot/ServiceWorker#method-unregister">unregistered</a> using the Chrome Devtools Protocol. Service workers need to be unregistered <em>per page request</em>, so the protocol will need to be called for each iteration.</p> <p>Whether you want cached or first view speed metrics is dependant on the type of your application and how your users use it. For applications with heavy repeat usage, it&apos;s likely knowing cached times will be a better measurement of real world usage.</p> <pre><code class="language-javascript"><span class="keyword">const</span> metrics = []; <span class="comment">// array for collecting metrics per iteration</span> <span class="keyword">const</span> runs = <span class="number">10</span>; <span class="keyword">const</span> useCache = <span class="literal">false</span>; <span class="keyword">const</span> useServiceWorker = <span class="literal">false</span>; <span class="keyword">await</span> page.setCacheEnabled(useCache); <span class="keyword">async</span> <span class="function"><span class="keyword">function</span> <span class="title">disableServiceWorker</span>(<span class="params">disable</span>) </span>{ <span class="keyword">if</span>(disable) { <span class="keyword">await</span> client.send(<span class="string">&apos;ServiceWorker.enable&apos;</span>); <span class="keyword">await</span> client.send(<span class="string">&apos;ServiceWorker.unregister&apos;</span>, { <span class="attr">scopeURL</span>: <span class="keyword">new</span> URL(page.url()).origin }); } } <span class="keyword">for</span>(<span class="keyword">let</span> i = <span class="number">0</span>; i &lt; runs; i++) { <span class="keyword">await</span> disableServiceWorker(!useServiceWorker); <span class="comment">// capture metrics per iteration</span> }</code></pre> <div class="demo-link"> <a href="https://github.com/scurker/page-speed-metrics-with-puppeteer/blob/master/examples/disable-caching.js">view disable cache example</a> </div> <div class="demo-link"> <a href="https://github.com/scurker/page-speed-metrics-with-puppeteer/blob/master/examples/disable-service-worker.js">view disable service worker example</a> </div> <h3 id="tracking-page-metrics"><a class="anchor-link" aria-hidden="true" href="#tracking-page-metrics"></a>Tracking Page Metrics</h3> <p>Page speed metrics need to be captured over multiple iterations to minimize any natural variations that might occur. We can collect these metrics using the <a href="https://developer.mozilla.org/en-US/docs/Web/API/Performance">Performance API</a>. This allows us to collect the native measurements we&apos;re interesting in, such as navigation and paint as well as custom measurements.</p> <p>To track custom measurements, you will need to create measurements with <a href="https://developer.mozilla.org/en-US/docs/Web/API/Performance/mark"><code>Performance.mark()</code></a> or <a href="https://developer.mozilla.org/en-US/docs/Web/API/Performance/mark"><code>Performance.measure()</code></a>.</p> <h4 id="tracking-first-meaningful-paint"><a class="anchor-link" aria-hidden="true" href="#tracking-first-meaningful-paint"></a>Tracking First Meaningful Paint</h4> <p>Since there is no standardized measurement for <abbr title="First meaningful paint">FMP</abbr>, you will need to manually include a performance mark to indicate where the first meaningful paint should happen.</p> <pre><code>&lt;<span class="keyword">div</span> <span class="built_in">class</span>=<span class="string">&quot;hero&quot;</span>&gt; ... &lt;/<span class="keyword">div</span>&gt; &lt;<span class="keyword">script</span>&gt; performance.mark(&apos;<span class="keyword">first</span>-meaningful-paint&apos;); &lt;/<span class="keyword">script</span>&gt;</code></pre><h4 id="tracking-time-to-interactive"><a class="anchor-link" aria-hidden="true" href="#tracking-time-to-interactive"></a>Tracking Time to Interactive</h4> <p>While puppeteer doesn&apos;t provide <abbr title="Time to interactive">TTI</abbr>, there are other things that you can track that can still provide some indication of interactiveness. If you&apos;re using a javascript library such as React or Vue, you could use lifecycle hooks inside of an important component to indicate that things are ready.</p> <blockquote> <p>React Example</p> </blockquote> <pre><code class="language-javascript"><span class="class"><span class="keyword">class</span> <span class="title">ImportantComponent</span> <span class="keyword">extends</span> <span class="title">React</span>.<span class="title">Component</span> </span>{ componentDidMount() { performance.mark(<span class="string">&apos;important component mounted&apos;</span>); } }</code></pre> <blockquote> <p>Vue Example</p> </blockquote> <pre><code class="language-javascript"><span class="keyword">export</span> <span class="keyword">default</span> { mounted() { <span class="keyword">this</span>.$nextTick(<span class="function"><span class="params">()</span> =&gt;</span> performance.mark(<span class="string">&apos;important component mounted&apos;</span>)); } }</code></pre> <p>Since these measurements will largely be app specific, there&apos;s no standard way to set where these measurements should be placed.</p> <p>With puppeteer, we can use <a href="https://github.com/GoogleChrome/puppeteer/blob/master/docs/api.md#pageevaluatepagefunction-args"><code>page.evaluate()</code></a> to run a function inside the context of the page. This can be used to access the performance api and return the results to our script.</p> <p>In addition, for each iteration we likely want to wait some time between requests to not introduce significant load to the server. Using <code>page.waitFor()</code> between iterations will allow for some time to ease the load.</p> <pre><code class="language-javascript"><span class="keyword">for</span>(<span class="keyword">let</span> i = <span class="number">0</span>; i &lt; runs; i++) { <span class="keyword">await</span> disableServiceWorker(!useServiceWorker); <span class="comment">// Allow time between requests</span> <span class="keyword">await</span> page.waitFor(<span class="number">500</span>); <span class="keyword">await</span> page.reload({ <span class="attr">waitUntil</span>: <span class="string">&apos;networkIdle0&apos;</span> }); <span class="keyword">let</span> navigationMetrics = <span class="built_in">JSON</span>.parse( <span class="keyword">await</span> page.evaluate(<span class="function"><span class="params">()</span> =&gt;</span> <span class="built_in">JSON</span>.stringify(performance.getEntriesByType(<span class="string">&apos;navigation&apos;</span>)) ) ); <span class="keyword">let</span> firstContentfulPaint = <span class="built_in">JSON</span>.parse( <span class="keyword">await</span> page.evaluate(<span class="function"><span class="params">()</span> =&gt;</span> <span class="built_in">JSON</span>.stringify(performance.getEntriesByName(<span class="string">&apos;first-contentful-paint&apos;</span>)) ) ); <span class="keyword">let</span> firstMeaningfulPaint = <span class="built_in">JSON</span>.parse( <span class="keyword">await</span> page.evaluate(<span class="function"><span class="params">()</span> =&gt;</span> <span class="built_in">JSON</span>.stringify(performance.getEntriesByName(<span class="string">&apos;first-meaningful-paint&apos;</span>)) ) ); <span class="comment">// any other additional custom metrics can be included and tracked</span> metrics.push({ <span class="attr">responseEnd</span>: navigationMetrics[<span class="number">0</span>].responseEnd, <span class="attr">loaded</span>: navigationMetrics[<span class="number">0</span>].domContentLoadedEventEnd, <span class="attr">complete</span>: navigationMetrics[<span class="number">0</span>].domComplete, <span class="attr">firstContentfulPaint</span>: firstContentfulPaint[<span class="number">0</span>].startTime, <span class="attr">firstMeaningfulPaint</span>: firstMeaningfulPaint[<span class="number">0</span>].startTime }); } <span class="comment">// Do something with the results</span> <span class="keyword">await</span> browser.close();</code></pre> <span class="demo-link"> <a href="https://github.com/scurker/page-speed-metrics-with-puppeteer/blob/master/examples/everything.js">view full puppeteer example</a> </span> <p>Once you&apos;ve collected all of the metric results that your heart desires, be sure to call <code>browser.close()</code> to ensure you don&apos;t keep your Puppeteer instance running.</p> <h3 id="analyzing-puppeteers-results"><a class="anchor-link" aria-hidden="true" href="#analyzing-puppeteers-results"></a>Analyzing Puppeteer&apos;s Results</h3> <p>There&apos;s a number of things you could do to analyze the results from Puppeteer:</p> <ul> <li>Toggle service worker on/off to see impact on repeat requests</li> <li>Add additional custom metrics to see more granular results</li> <li>Test against multiple network profiles</li> </ul> <p>Since you have near complete control over Chrome using puppeteer&apos;s api, there&apos;s few limitations in what speed metrics you can track provided you have placed useful marks in places that matter.</p> <h2 id="measure-measure-measure-and-measure-again"><a class="anchor-link" aria-hidden="true" href="#measure-measure-measure-and-measure-again"></a>Measure, Measure, Measure (and measure again)</h2> <p>Knowing how fast or slow your site is important, and it&apos;s impossible to know without continually measuring and monitoring changes. For users that use your site every day, they feel pain when things don&apos;t appear to be responsive. The tools listed above are by no means an exhaustive list, but are just a starting point to begin having a better understanding of the different metrics that affect your users. From this, hopefully you will feel more empowered to start tracking and measuring leading to a better experience for all of your users.</p></content></entry><entry><title>Clearing a Path for Offline</title><link href="https://scurker.com/clearing-a-path-for-offline"/><id>https://scurker.com/clearing-a-path-for-offline</id><updated>2018-06-05T00:00:00+00:00</updated><content type="html"><p>I&apos;ve long known the benefits of service workers, but had not made the push to implement one for my own site. With the recent <a href="https://webkit.org/blog/8090/workers-at-your-service/">change in Safari</a> finally taking service workers to task, it was time to hop aboard the service worker train myself. Now if you visit my site, select pages will be available with offline viewing.</p> <p>As of <a href="https://caniuse.com/#feat=serviceworkers">June 2018</a>, 83% of global users have some form of service worker support so there&apos;s little reason not to have one especially as mobile connectivity becomes more pervasive. If you&apos;ve made it this far and still don&apos;t know what a service worker is, <a href="https://developers.google.com/web/fundamentals/primers/service-workers/">Google has an excellent primer</a> all about the inner workings of service workers.</p> <h2 id="a-worker-to-build"><a class="anchor-link" aria-hidden="true" href="#a-worker-to-build"></a>A worker to build</h2> <p>I build my blog on top of <a href="http://www.metalsmith.io/">metalsmith</a>, and there was a couple of key aspects I wanted to build into my service worker:</p> <ul> <li>Increment cache version for each static site build</li> <li>Have my service worker automatically include built static assets</li> <li>Have my offline page list previously visited offline pages</li> </ul> <p>I am not going to suggest the path that I took towards the end result is one that should be similarly followed, but wanted to outline the steps that led me there.</p> <h3 id="versioning"><a class="anchor-link" aria-hidden="true" href="#versioning"></a>Versioning</h3> <p>Whenever a service worker changes, a new version is downloaded and installed. To ensure that the browser&apos;s cache is up to date we want the service worker to have a new version key to be able to clear out older requests.</p> <p>My static site doesn&apos;t change frequently, so having a cache version based on date seems relatively safe. Using <a href="https://www.npmjs.com/package/rollup">rollup</a> in combination with <a href="https://www.npmjs.com/package/rollup-plugin-replace">rollup-plugin-replace</a> I can replace <code>CACHE_VERSION</code> with the current date&apos;s version string.</p> <pre><code><span class="selector-tag">replace</span>({ <span class="attribute">CACHE_VERSION</span>: JSON.<span class="built_in">stringify</span>( new Date().<span class="built_in">toISOString</span>().<span class="built_in">split</span>(<span class="string">&apos;T&apos;</span>)[<span class="number">0</span>].<span class="built_in">replace</span>(/-/g, <span class="string">&apos;&apos;</span>) ) })</code></pre><h3 id="including-assets"><a class="anchor-link" aria-hidden="true" href="#including-assets"></a>Including Assets</h3> <p>Similarly to versioning above, I utilize rollup to scan my assets folder to match static assets and replace <code>STATIC_ASSETS</code> with the array of assets to pre-cache in the service worker on install.</p> <pre><code>replace({ <span class="attr">STATIC_ASSETS</span>: <span class="built_in">JSON</span>.stringify([ ...glob.sync(<span class="string">&apos;assets/**/*&apos;</span>, { <span class="attr">cwd</span>: path.resolve(__dirname, <span class="string">&apos;build&apos;</span>), <span class="attr">nodir</span>: <span class="literal">true</span> }).map(<span class="function"><span class="params">file</span> =&gt;</span> <span class="string">`/<span class="subst">${file}</span>`</span>) ], <span class="literal">null</span>, <span class="number">2</span>) })</code></pre><h3 id="listing-offline-posts"><a class="anchor-link" aria-hidden="true" href="#listing-offline-posts"></a>Listing offline posts</h3> <p>For listing offline posts, I&apos;m utilizing <a href="https://www.npmjs.com/package/metalsmith#useplugin">metalsmith middleware</a> to generate <a href="https://scurker.com/pages.json">json metadata</a> about my site to pre-cache in my service worker to later compare cached requests against known pages. I could just list visited offline posts by their url, but by generating and pre-caching metadata about any particular page, I can include page titles and dates to list <a href="/offline">/offline</a>.</p> <pre><code class="language-js"><span class="keyword">const</span> isOfflinePage = <span class="built_in">document</span>.body.classList.contains(<span class="string">&apos;offline-page&apos;</span>); <span class="keyword">if</span> (<span class="built_in">window</span>.caches &amp;&amp; isOfflinePage) { caches.keys() .then(<span class="function"><span class="params">keys</span> =&gt;</span> { <span class="keyword">return</span> <span class="built_in">Promise</span>.all([ caches.open(targetCache).then(<span class="function"><span class="params">c</span> =&gt;</span> c.keys()), fetch(<span class="string">&apos;/pages.json&apos;</span>).then(<span class="function"><span class="params">res</span> =&gt;</span> res.json()) ]); }) .then(<span class="function">(<span class="params">[ requests, pages ]</span>) =&gt;</span> { <span class="comment">// match the requests in the cache against known posts</span> <span class="keyword">let</span> posts = pages.filter(<span class="function">(<span class="params">{ type }</span>) =&gt;</span> type === <span class="string">&apos;post&apos;</span>) , matchedPosts = posts.filter( <span class="function"><span class="params">post</span> =&gt;</span> requests.find(<span class="function"><span class="params">req</span> =&gt;</span> <span class="keyword">new</span> URL(req.url).pathname.indexOf(post.path) !== <span class="number">-1</span>) ); <span class="keyword">let</span> offlineContainer = <span class="built_in">document</span>.querySelector(<span class="string">&apos;.offline-posts&apos;</span>); <span class="keyword">if</span>(matchedPosts.length) { <span class="comment">// display offline posts on the page</span> } <span class="keyword">else</span> { offlineContainer.remove(); } }); }</code></pre> <h2 id="there-are-many-service-workers--this-one-is-mine"><a class="anchor-link" aria-hidden="true" href="#there-are-many-service-workers--this-one-is-mine"></a>There are many service workers &#x2013; this one is mine</h2> <p>Beyond the methods outline above, my service worker isn&apos;t a particularly specialized affair. Following patterns from <a href="https://developers.google.com/web/fundamentals/instant-and-offline/offline-cookbook">Jake Archibald&apos;s offline cookbook</a>, <a href="https://developers.google.com/web/fundamentals/instant-and-offline/offline-cookbook/#on-install-not">pre-caches assets on install</a>, <a href="https://developers.google.com/web/fundamentals/instant-and-offline/offline-cookbook/#on-activate">removes old caches on activate</a>, and serving content with a <a href="https://developers.google.com/web/fundamentals/instant-and-offline/offline-cookbook/#network-falling-back-to-cache">network first with cache fallback</a> pattern.</p> <p>For a full picture, you can view my service worker scripts below.</p> <ul> <li><a href="https://github.com/scurker/scurker.com/blob/master/site/sw.js"><code>sw.js</code></a> <em>prebuild</em></li> <li><a href="https://scurker.com/sw.js"><code>sw.js</code></a> <em>postbuild</em></li> <li><a href="https://github.com/scurker/scurker.com/blob/master/rollup.sw.config.js"><code>rollup.sw.config.js</code></a></li> </ul> <h2 id="thoughts-to-improve"><a class="anchor-link" aria-hidden="true" href="#thoughts-to-improve"></a>Thoughts to improve</h2> <p>These are only my first steps towards a path for offline, but there are certain areas of improvement I could see. Service worker asset responses could follow a <a href="https://developers.google.com/web/fundamentals/instant-and-offline/offline-cookbook/#cache-falling-back-to-network">cache with network fallback</a> recipe since my assets aren&apos;t likely to change frequently. Or, if I was to uniquely identify assets with some sort of asset hash could place assets into a more permanent cache only expiring assets if a new version is received.</p></content></entry><entry><title>Interesting talks at JSConf 2015</title><link href="https://scurker.com/interesting-talks-at-jsconf-2015"/><id>https://scurker.com/interesting-talks-at-jsconf-2015</id><updated>2015-05-31T00:00:00+00:00</updated><content type="html"><p>This is my second year of being able to attend JSConf and feel fortunate to be able to listen and interact with so many awesome people in the community. This is mostly a brain dump of some of the topics I found interesting at JSConf, but obviously isn&apos;t every talk or speaker since I couldn&apos;t attend every session. Hopefully, some others can benefit from some of the knowledge I gained - or find some interesting people to follow in the community.</p> <h2 id="js-accessibility-and-js-side-by-side---felipe-de-albuquerque"><a class="anchor-link" aria-hidden="true" href="#js-accessibility-and-js-side-by-side---felipe-de-albuquerque"></a>JS Accessibility and JS: side-by-side - <a href="https://twitter.com/felipedeolinda">Felipe de Albuquerque</a></h2> <p>This was an interesting talk on accessibility because the talk itself wasn&apos;t presented in English. Felipe gave a few examples of why accessibility is important:</p> <ul> <li>650 million people have some some kind of disability</li> <li>Kids can control websites through voice before they can read, so having voice control would allow young kids to navigate</li> <li>The internet should be inclusive of everyone</li> <li>Based on research, <a href="http://www.freedomscientific.com/Products/Blindness/JAWS">JAWS</a> is one of the most popular screen readers</li> </ul> <p>One of the biggest issues we have in the community is the lack of developer knowledge with WAI-ARIA attributes. ARIA attributes are something that are ready to be implemented <em>today</em>, and as a community we should be concerned about making our applications accessible to everyone.</p> <h2 id="communicate-all-the-things---kyle-tyacke"><a class="anchor-link" aria-hidden="true" href="#communicate-all-the-things---kyle-tyacke"></a>Communicate All the Things - <a href="https://twitter.com/geekgonenomad">Kyle Tyacke</a></h2> <p>Building the web with WebRTC. If you&apos;ve ever used Amazon Mayday, or Google Hangouts you&apos;ve used WebRTC. WebRTC is similar to web sockets, but uses peer-to-peer connections instead of communicating with a server.</p> <p><a href="https://github.com/respoke/apollo">Apollo</a> is a JavaScript library built on <a href="https://www.respoke.io/">respoke</a> that helps to handle those peer-to-peer connections. From there you can create peer-to-peer chat or video connections directly in your application.</p> <h2 id="async-programing-in-es7-es2016---jafar-husain"><a class="anchor-link" aria-hidden="true" href="#async-programing-in-es7-es2016---jafar-husain"></a>Async Programing in <del>ES7</del> ES2016 - <a href="http://twitter.com/Jhusain">Jafar Husain</a></h2> <p>How do we make async easier? What if you could write async programs without any callbacks at all? What if waiting was just as easy as creating blocking functions?</p> <p>With ES2016, you can use generators to write blocking or asynchronous code exactly the same:</p> <pre><code class="language-javascript"><span class="keyword">async</span> <span class="function"><span class="keyword">function</span> <span class="title">foo</span>(<span class="params"></span>) </span>{ <span class="keyword">var</span> bar = <span class="keyword">await</span> foobar(); <span class="keyword">return</span> bar; }</code></pre> <p>Currently ES2016 is in the draft stage, but it&apos;s expected to be part of the standard.</p> <p>This was a great talk that helped explain some of the features <del>ES5</del>ES2015 on generators and how they differ from iterators. One great point is that iterators are used for one-way functional communication, while generators are used for two-way functional communication. By calling <code>result.next(&quot;value&quot;)</code> on a generator, you can push values to a generator function.</p> <p>It&apos;s hard to explain everything in short, but Jafar has created a repository that goes into further detail on <a href="https://github.com/jhusain/asyncgenerator">generators, observables, and async generators</a>.</p> <h2 id="knitting-for-javascript---mariko-kosaka"><a class="anchor-link" aria-hidden="true" href="#knitting-for-javascript---mariko-kosaka"></a>Knitting for JavaScript - <a href="https://twitter.com/kosamari">Mariko Kosaka</a></h2> <p>Apparently there&apos;s a &quot;Github&quot; for knitters, <a href="https://www.ravelry.com">Ravelry</a>.</p> <p>Code for knitting.</p> <pre><code><span class="built_in">R0</span> : <span class="built_in">k5</span> <span class="built_in">R1</span>-<span class="number">3</span> : <span class="built_in">k2</span>, m1, k until <span class="number">2</span> sts remain, m1, <span class="built_in">k2</span> <span class="built_in">R4</span> : <span class="built_in">k2</span>, m1, <span class="built_in">k1</span>, m1, k until <span class="number">3</span> sts remain, m1, <span class="built_in">k1</span>, m1, <span class="built_in">k2</span> Repeat <span class="built_in">R1</span>-<span class="number">4</span> for <span class="number">10</span> <span class="built_in">times</span></code></pre><p>That looks like something that could easily be created in javascript.</p> <p>Mariko took two seemingly unrelated topics and merged them together to create something amazing and fascinating. This was a great talk that helped expose how JavaScript could be used for things that no one would really expect.</p> <p>Kariko plans on putting up some of her code on Github at <a href="https://github.com/kosamari/electroknit">electroknit</a> and <a href="https://github.com/kosamari/color-mixer">color-mixer</a>.</p> <h2 id="30-minutes-or-less-the-magic-of-automated-accessibility-testing---marcy-sutton"><a class="anchor-link" aria-hidden="true" href="#30-minutes-or-less-the-magic-of-automated-accessibility-testing---marcy-sutton"></a>30 Minutes or Less: The Magic of Automated Accessibility Testing - <a href="https://twitter.com/marcysutton">Marcy Sutton</a></h2> <p>What are some the the accessibility basics you should be aware of?</p> <ul> <li>Alternative text</li> <li>Document structure &amp; hierarchy</li> <li>HTML Semantics</li> <li>Keyboard interactivity</li> <li>Color contrast</li> <li>Focus management</li> </ul> <p>Let the tooling do the heavy lifting for you! Tools can help you identify some of your accessibility issues.</p> <p>In Chrome Canary, you can enable <a href="https://github.com/GoogleChrome/accessibility-developer-tools">accessibility developer tools</a> that will allow you to run audits on your page to help find those issues.</p> <ul> <li><a href="https://www.npmjs.com/package/a11y">A11Y (Ally)</a> - run accessibility audits against a site.</li> <li><a href="http://marcysutton.com/angular-protractor-accessibility-plugin/">Protractor + Accessibility Plugin</a> - end to end testing accessibility plugin Angular JS</li> </ul></content></entry><entry><title>In Harmony with Javascript - ES6, Javascript, and You</title><link href="https://scurker.com/in-harmony-with-javascript-es6-javascript-and-you"/><id>https://scurker.com/in-harmony-with-javascript-es6-javascript-and-you</id><updated>2015-04-25T00:00:00+00:00</updated><content type="html"><p>I have been wanting to give a talk on some of the exciting features of ES6 Harmony - and how you could start utilizing certain components of ES6 today, and <a href="http://www.barcampbham.com/">BarCamp Birmingham</a> was a great venue to introduce this talk.</p> <p>It&apos;s hard to talk about <a href="https://github.com/lukehoban/es6features">all the great features</a> that are a part of ES6, but I mainly wanted to touch on the ones that are usable today and the various strategies for implementing ES6 features as part of your code.</p> <p>Want to know the short of it? For browsers, it&apos;s fairly safe to use many of the ES6 features outlined below with the right polyfill. For node or io, io has better support out of the box while node 0.11.x+ requires specific ES6 feature flags enabled.</p> <div class="video-wrapper"> <iframe src="//slides.com/scurker/es6-javascript-and-you/embed?style=light" width="576" height="420" scrolling="no" frameborder="0" webkitallowfullscreen mozallowfullscreen allowfullscreen></iframe> </div> <p>Important links</p> <ul> <li><a href="https://github.com/lukehoban/es6features">ES6 Complete Feature Set</a></li> <li><a href="https://status.modern.ie/?term=es6">Internet Explorer ES6 Support</a> on modern.ie</li> <li><a href="https://kangax.github.io/compat-table/es6/">ES6 Compatibility Tables</a> including browsers and javascript engines</li> <li><a href="https://github.com/zloirock/core-js">core-js</a>, various polyfills for ES5, ES6, and ES7</li> <li><a href="https://github.com/paulmillr/es6-shim/">es6-shim</a> compatibility shims for ES6</li> <li><a href="https://babeljs.io/">Babel</a> a transpiler for compiling ES6 code to ES5</li> <li><a href="https://github.com/google/traceur-compiler">Traceur</a> compiles ES6 as well as some ES.next features</li> </ul> <p>You can view the <a href="http://slides.com/scurker/es6-javascript-and-you">full sets of slides over at slides.com</a>.</p></content></entry><entry><title>Automated Deploys with Travis CI</title><link href="https://scurker.com/automated-deploys-with-travis-ci"/><id>https://scurker.com/automated-deploys-with-travis-ci</id><updated>2015-03-09T00:00:00+00:00</updated><content type="html"><p>If you read my previous post about <a href="/remote-deployments-with-git">remote deployments with git</a>, you should have a good idea on how you can deploy with a simple git push. This method works fine if you only work from one computer but starts to be a little more difficult if you want to use other devices (such as an tablet or phone) and potentially want to write posts or content directly on Github.</p> <h2 id="bring-in-travis-ci"><a class="anchor-link" aria-hidden="true" href="#bring-in-travis-ci"></a>Bring in Travis CI</h2> <p>If you aren&apos;t using a CI (continuous integration) tool as part of your workflow, you should be! I&apos;ve started to become a big fan of <a href="https://travis-ci.org">Travis CI</a> because it&apos;s easy to setup, integrates easily with GitHub, and runs on a <a href="http://docs.travis-ci.com/user/getting-started/#Travis-CI-Overview">number of different languages</a>. And best of all, it&apos;s free<sup>*</sup>!</p> <p><small>* for open source projects</small></p> <p>I use Travis to centralize my pushes so that I no longer have to have setup remote pushes from my local computer directly to my remote server to update content, but rather push directly to a Github branch and let Travis take care of the rest.</p> <h2 id="getting-started"><a class="anchor-link" aria-hidden="true" href="#getting-started"></a>Getting Started</h2> <ul> <li>Remote server for hosting your site</li> <li>Create limited access git user (recommended)</li> <li>Create public/private SSH key pairs for authentication</li> <li>Add public key to authorized SSH keys</li> <li>Add private key to repository</li> <li>Setup Travis</li> <li>Setup travis.yml</li> </ul> <h2 id="generate-ssh-key-pairs"><a class="anchor-link" aria-hidden="true" href="#generate-ssh-key-pairs"></a>Generate SSH key pairs</h2> <p>Since I don&apos;t exactly feel comfortable giving Travis or Github my personal credentials to my own server, I&apos;m using SSH authentication in order to grant access to push. In addition, I&apos;ve created a <a href="http://git-scm.com/book/en/v2/Git-on-the-Server-Setting-Up-the-Server">limited access git user</a> that only has access to my git and web directories -- nothing else.</p> <p>I&apos;m not going to go into great detail on how to create your SSH key pairs, since Github has <a href="https://help.github.com/articles/generating-ssh-keys/">already done that for us</a>. You will need to login as your git user on your remote server, and generate a key pair following the above steps.</p> <p>Once you&apos;ve followed those steps, you should have two files <code>~/.ssh/id_rsa</code>, your private key and <code>~/.ssh/id_rsa.pub</code>, your public key. In order for the SSH authentication to work properly you will need to copy your public key into an authorized keys file.</p> <pre><code class="language-bash">cat ~/.ssh/id_rsa.pub &gt;&gt; ~/.ssh/authorized_keys</code></pre> <h2 id="add-private-key-to-repository"><a class="anchor-link" aria-hidden="true" href="#add-private-key-to-repository"></a>Add Private Key to Repository</h2> <p>Now that you&apos;ve created a SSH key pair, you will need to copy your private key so that Travis can use it to authenticate to your remote server. Since we don&apos;t want to be putting our <a href="http://www.devfactor.net/2014/12/30/2375-amazon-mistake/">private key unencrypted</a> into Github, we&apos;ll be using Travis to encrypt the key before we push it out for everyone to see. You will need to install either the Travis ruby gem, or travis-encrypt from npm if you haven&apos;t already done so.</p> <h3 id="ruby"><a class="anchor-link" aria-hidden="true" href="#ruby"></a>Ruby</h3> <pre><code class="language-ruby">gem install travis</code></pre> <h3 id="npm"><a class="anchor-link" aria-hidden="true" href="#npm"></a>NPM</h3> <pre><code>npm <span class="keyword">install</span> travis-<span class="keyword">encrypt</span></code></pre><p>I&apos;m personally using the official Travis ruby gem, but the rest of the arguments should be about the same for both. You can read more about <a href="http://docs.travis-ci.com/user/encryption-keys/">Travis encryption</a>, but I&apos;m going to give you the Cliff Notes&#x2122; version here.</p> <p>Assuming you&apos;re already in your project directory, it&apos;s as easy as...</p> <pre><code class="language-bash">mv id_rsa deploy_key touch .travis.yml &amp;&amp; travis encrypt-file deploy_key --add</code></pre> <p>This initializes the <code>.travis.yml</code> file, encrypts your deploy key, and adds all the necessary information for Travis to be able to use that encrypted file.</p> <p>You&apos;ll need to commit your changes to Github, but before you do so remember to remove your private key <code>deploy_key</code> and store it somewhere safe!</p> <h2 id="setup-travis"><a class="anchor-link" aria-hidden="true" href="#setup-travis"></a>Setup Travis</h2> <p>From here, you&apos;ll want to to go to <a href="https://travis-ci.org/">travis-ci.org</a>, authenticate with Github, and turn on Travis for the repository that you wish to start deploying.</p> <p>From here, you&apos;ll need to make a few additions to your <code>.travis.yml</code> file, and I&apos;ll include an example, but you can of course always view the <a href="https://github.com/scurker/scurker.com/blob/master/.travis.yml">latest version</a> on Github.</p> <pre><code class="language-yaml"><span class="attr">language:</span> <span class="string">node_js</span> <span class="attr">node_js:</span> <span class="string">&apos;0.10&apos;</span> <span class="attr">install:</span> <span class="string">echo</span> <span class="string">&quot;skip install&quot;</span> <span class="attr">branches:</span> <span class="attr"> only:</span> <span class="string">master</span> <span class="attr">after_success:</span> <span class="bullet">-</span> <span class="string">chmod</span> <span class="number">600</span> <span class="string">deploy-key</span> <span class="bullet">-</span> <span class="string">mv</span> <span class="string">deploy-key</span> <span class="string">~/.ssh/id_rsa</span> <span class="bullet">-</span> <span class="string">git</span> <span class="string">remote</span> <span class="string">add</span> <span class="string">deploy</span> <span class="attr">ssh://git@example.com/var/git/site.git</span> <span class="bullet">-</span> <span class="string">git</span> <span class="string">push</span> <span class="string">deploy</span> <span class="attr">before_install:</span> <span class="bullet">-</span> <span class="string">echo</span> <span class="bullet">-e</span> <span class="string">&quot;Host example.com\n\tStrictHostKeyChecking no\n&quot;</span> <span class="string">&gt;&gt;</span> <span class="string">~/.ssh/config</span> <span class="bullet">-</span> <span class="string">openssl</span> <span class="string">aes-256-cbc</span> <span class="bullet">-K</span> <span class="string">$encrypted_6a5cf90fd664_key</span> <span class="bullet">-iv</span> <span class="string">$encrypted_6a5cf90fd664_iv</span> <span class="bullet"> -</span><span class="string">in</span> <span class="string">deploy-key.enc</span> <span class="bullet">-out</span> <span class="string">deploy-key</span> <span class="bullet">-d</span></code></pre> <p>You&apos;ll need to change things accordingly to your needs, but here&apos;s a couple of tips:</p> <pre><code class="language-yaml"><span class="attr">install:</span> <span class="string">echo</span> <span class="string">&quot;skip install&quot;</span></code></pre> <p>I&apos;m essentially skipping this step because I&apos;m only using Travis to deploy, but theoretically, you could use this to run tests before the deployment actually runs.</p> <pre><code class="language-yaml"><span class="attr">after_success:</span> <span class="bullet">-</span> <span class="string">chmod</span> <span class="number">600</span> <span class="string">deploy-key</span> <span class="bullet">-</span> <span class="string">mv</span> <span class="string">deploy-key</span> <span class="string">~/.ssh/id_rsa</span> <span class="bullet">-</span> <span class="string">git</span> <span class="string">remote</span> <span class="string">add</span> <span class="string">deploy</span> <span class="attr">ssh://git@example.com/var/git/site.git</span> <span class="bullet">-</span> <span class="string">git</span> <span class="string">push</span> <span class="string">deploy</span></code></pre> <p>Here&apos;s where the bulk of the work actually happen. Your private key is copied to the appropriate directory, and the remote origin is created and pushed to.</p> <pre><code>before_install: -<span class="ruby"> echo -e <span class="string">&quot;Host example.com\n\tStrictHostKeyChecking no\n&quot;</span> <span class="meta">&gt;&gt; </span>~<span class="regexp">/.ssh/config</span></span></code></pre><p>You will also need to change this to your domain as well, otherwise Travis may not be able to authenticate the domain and will sit there until the job times out.</p> <p>With this, any time I push to Github Travis is able to deploy those changes and immediately push them out to this blog. You can always view <a href="https://github.com/scurker/scurker.com">everything on Github</a>.</p></content></entry><entry><title>Remote Deployments with Git</title><link href="https://scurker.com/remote-deployments-with-git"/><id>https://scurker.com/remote-deployments-with-git</id><updated>2015-01-25T00:00:00+00:00</updated><content type="html"><p>I&apos;ve been deploying using git for my static and node sites for several months and wanted to offer some tips on how you can do the same. This is very similar to <a href="https://devcenter.heroku.com/articles/git">heroku&apos;s deployment model</a> but using your own server in place of theirs.</p> <h3 id="getting-started"><a class="anchor-link" aria-hidden="true" href="#getting-started"></a>Getting Started</h3> <ul> <li>Remote server for hosting your site</li> <li>SSH access on remote server</li> <li>Git installed locally and remotely</li> <li>(optional) npm installed remotely</li> <li>(optional) grunt installed remotely</li> </ul> <p>For the remote server, you&apos;ll need two different directories to store your files. One is where your site will be hosted, the other will be used for the git repository. I&apos;m using <code>/var/www</code> and <code>/var/git</code> respectively, but you can choose differently based on your setup.</p> <h3 id="create-remote-repository"><a class="anchor-link" aria-hidden="true" href="#create-remote-repository"></a>Create remote repository</h3> <p>For the remote server, you&apos;ll need to start with a bare git repository. This is where you will push your content to from your local machine.</p> <pre><code class="language-bash"><span class="built_in">cd</span> /var/git mkdir site.git &amp;&amp; <span class="built_in">cd</span> site.git git init --bare</code></pre> <h3 id="remote-dependencies"><a class="anchor-link" aria-hidden="true" href="#remote-dependencies"></a>Remote dependencies</h3> <p>Whatever you&apos;re using to build locally, you&apos;ll also need to be sure those dependencies are installed on your remote server. I&apos;m using <a href="https://www.npmjs.com/">npm</a> and <a href="http://gruntjs.com/">grunt</a> to build everything - but this could be ruby, jekyll or anything used to compile your site.</p> <p>Since I&apos;m using grunt, we&apos;ll need to be sure that grunt-cli is installed globally.</p> <pre><code class="language-bash">npm install -g grunt-cli</code></pre> <h3 id="create-post-receive-hook"><a class="anchor-link" aria-hidden="true" href="#create-post-receive-hook"></a>Create post receive hook</h3> <p>We&apos;ll be using a <a href="http://git-scm.com/docs/githooks#post-receive">post receive hook</a> to run tasks after commits are received, such as building the site. Since your git directory probably isn&apos;t where you&apos;re serving up your files, you can use <code>--work-tree</code> to set the working directory for your committed files.</p> <h4 id="post-receive-hook"><a class="anchor-link" aria-hidden="true" href="#post-receive-hook"></a>post-receive hook</h4> <pre><code class="language-bash"><span class="meta">#!/bin/sh </span> <span class="comment"># Use a build path as a temp staging directory before copying over</span> <span class="comment"># to the &quot;live&quot; web directory</span> BUILD_PATH=$(<span class="built_in">cd</span> <span class="string">&quot;<span class="variable">$(dirname &quot;$0&quot;)</span>/..&quot;</span>; <span class="built_in">pwd</span>)/build WEB_PATH=/var/www/example.com <span class="comment"># Checkout latest version of files and cleanup untracked files</span> git --work-tree=<span class="variable">$BUILD_PATH</span> checkout -f git --work-tree=<span class="variable">$BUILD_PATH</span> clean -fd <span class="built_in">cd</span> <span class="variable">$BUILD_PATH</span>; npm install --production <span class="comment"># Use grunt to run a build task, but this could be anything you</span> <span class="comment"># want to use to generate your static site and/or run your node project</span> grunt build <span class="comment"># Clean web directory and copy static files to web directory</span> (<span class="built_in">cd</span> <span class="variable">$WEB_PATH</span>;rm -rf *) (cp -r <span class="variable">$BUILD_PATH</span>/build/. <span class="variable">$WEB_PATH</span>)</code></pre> <p>Be sure you make the post-receive hook executable.</p> <pre><code class="language-bash">chmod +x hooks/post-receive</code></pre> <h3 id="set-up-push-to-deploy"><a class="anchor-link" aria-hidden="true" href="#set-up-push-to-deploy"></a>Set up Push to Deploy</h3> <p>Now that everything is configured remotely, you&apos;ll need to set up your local git to push to your remote server.</p> <pre><code class="language-bash">git remote add deploy ssh://user@example.com/var/git/site.git</code></pre> <p>Now, you can push files to your remote server with one simple command:</p> <pre><code class="language-bash">... git add . git commit -m <span class="string">&quot;Updating files&quot;</span> git push deploy</code></pre> <p>If everything worked successfully, you should be able to view your site and see updated changes.</p> <h4 id="add-ssh-key"><a class="anchor-link" aria-hidden="true" href="#add-ssh-key"></a>Add SSH key</h4> <p>Don&apos;t like typing your username and password every time you deploy? You can use a public SSH key to authenticate every time you do your push to deploy. If you haven&apos;t already generated your SSH keys, you can <a href="https://help.github.com/articles/generating-ssh-keys/">follow this guide through step 2</a>.</p> <p>From there, you can simply copy your public key to your server.</p> <pre><code class="language-bash">cat ~/.ssh/id_rsa.pub | ssh user@example.com <span class="string">&quot;cat &gt;&gt; ~/.ssh/authorized_keys&quot;</span></code></pre> <p>What if you could automate your deployments? You can continue on reading on how you can enhance the push to deploy by making your deployments <a href="/automated-deploys-with-travis">completely automated with Travis CI</a>.</p></content></entry><entry><title>Bring on a new scurker.com</title><link href="https://scurker.com/bring-on-a-new-scurker-com"/><id>https://scurker.com/bring-on-a-new-scurker-com</id><updated>2014-12-14T00:00:00+00:00</updated><content type="html"><p>Welcome to the redesign and relaunch of scurker.com! It&apos;s been far too long in my professional career that I have neglected this site, and felt that a new fresh coat of paint was long overdue.</p> <p>It&apos;s also been three years since I have contributed anything back to the community, and part of the goal for this relaunch is to better share what I know and have learned.</p> <h3 id="so-long-wordpress"><a class="anchor-link" aria-hidden="true" href="#so-long-wordpress"></a>So Long Wordpress!</h3> <p>Wordpress was a good start, but has finally outlived its usefulness and I&apos;ve moved on to greener pastures. I wanted something much more lightweight and with more friendly editing. I had initially looked at <a href="http://ghost.org">ghost</a> along with a few other blogging platforms, but eventually decided that I didn&apos;t want to be encumbered by unnecessary overhead of yet another blog platform.</p> <h3 id="hello-wintersmith"><a class="anchor-link" aria-hidden="true" href="#hello-wintersmith"></a>Hello Wintersmith</h3> <p>Bring on <a href="http://wintersmith.io/">wintersmith</a>, a nodejs static site generator. I don&apos;t hate myself, so I also am using <a href="https://github.com/tnguyen14/wintersmith-handlebars">wintersmith handlebars</a> over the built in Jade template engine. For my sanity and easier editing I can write every post or page in markdown - which wintersmith will automatically convert.</p> <h3 id="dependencies"><a class="anchor-link" aria-hidden="true" href="#dependencies"></a>Dependencies</h3> <ul> <li><a href="http://wintersmith.io/">wintersmith</a></li> <li><a href="https://github.com/tnguyen14/wintersmith-handlebars">wintersmith handlebars</a></li> <li><a href="http://gruntjs.com">grunt</a></li> <li><a href="http://www.google.com/fonts/specimen/Noto+Sans">noto sans font</a></li> </ul> <p>There&apos;s some slightly modified plugins to wintersmith to allow for article to article pagination and a more sensible permalink structure.</p> <h3 id="no-more-jquery"><a class="anchor-link" aria-hidden="true" href="#no-more-jquery"></a>No more jQuery</h3> <p>I&apos;m a big fan of jQuery, but I have been <a href="http://youmightnotneedjquery.com/">slowly moving away from having jQuery</a> as a dependency unless absolutely necessary. So you won&apos;t see very much jQuery used here.</p> <h3 id="open-sourced"><a class="anchor-link" aria-hidden="true" href="#open-sourced"></a>Open Sourced</h3> <p>I&apos;ve learned a lot through open source projects, and have spoken many times about the importance of open source. However, my own blog I kept private and didn&apos;t share what I knew. I figured it&apos;s time to start putting money where my mouth is and release the source for this site. You can check out everything on <a href="http://github.com/scurker/scurker.com">github</a>. If you come across any issues, be sure to <a href="https://github.com/scurker/scurker.com/issues">let me know</a>.</p></content></entry><entry><title>Go Mobile at TechMixer University</title><link href="https://scurker.com/go-mobile-at-techmixer-university"/><id>https://scurker.com/go-mobile-at-techmixer-university</id><updated>2011-09-25T00:00:00+00:00</updated><content type="html"><p>I will be presenting a session on mobile web development this coming Tuesday, on September 28th. Slides will be posted shortly after the session is finished.</p> <ul> <li>Learn about recent trends regarding the mobile web</li> <li>&quot;Rethink&quot; developing for mobile</li> <li>Dive into mobile touch events and more!</li> </ul> <p>Couldn&apos;t make the session? View the slides online at <a href="http://slides.scurker.com/gomobile">http://slides.scurker.com/gomobile</a>.</p> <h3 id="techmixer-university"><a class="anchor-link" aria-hidden="true" href="#techmixer-university"></a>TechMixer University</h3> <p>TechMixer University is an annual one-day conference that offers a full day of free technology training. This Birmingham, Alabama event is attended by the area&#x2019;s brightest and enriched technical professionals, including developers, database professionals, project managers, network professionals, IT directors/managers and C-level executives.</p> <p><a href="http://www.techbirmingham.com/university/">http://www.techbirmingham.com/university/</a></p></content></entry><entry><title>Amazon Trade-in Value Tracker</title><link href="https://scurker.com/amazon-trade-in-value-tracker"/><id>https://scurker.com/amazon-trade-in-value-tracker</id><updated>2011-04-13T00:00:00+00:00</updated><content type="html"><p>This was a project that actually begin back in the summer of &apos;09. At the time Amazon had introduced a trade-in system with some pretty tempting promotions. However, there were limitations in that you could only see the most recent trade-in value with no ability for a history or a way to detect changes. Thus, the ativ tracker was born.</p> <p>Over a year later and with nearly 700,000 values tracked, the tracker is still chugging along but needed a new coat of paint. It&apos;s been updated with a new theme, some background improvements, better mobile optimizations, and charting utilizing <a href="http://raphaeljs.com/">Rapha&#xEB;lJS</a>.</p> <p>A lot of sweat and hard work was put into this project to make it easy to track trade-in values. So if you have some old games sitting in your closet, why not consider <a href="http://tivtrackr.com">checking out the tracker</a> and turning your games into Amazon credit?</p></content></entry><entry><title>Scurker Levels Up</title><link href="https://scurker.com/scurker-levels-up"/><id>https://scurker.com/scurker-levels-up</id><updated>2010-12-14T00:00:00+00:00</updated><content type="html"><p>Welcome to the new and improved scurker.com! I&apos;d never felt comfortable with my previous design and never really felt inspired by it. There had been some things that I had felt had been missing, and the site lacked a lot of flexibility I was looking for. Shortly after my previous design had been launched, I begin work on this iteration giving time for things to settle in. What has changed?</p> <h3 id="standards-are-dead"><a class="anchor-link" aria-hidden="true" href="#standards-are-dead"></a>Standards Are Dead?</h3> <p>Well technically, no. Previously I have been using XHTML 1.0 strict, but have now switched to an HTML 5 doctype, which is simple as <code>&lt;!DOCTYPE html&gt;</code>. The majority of the features I plan on using are quite usable across most major browsers, so I see little reason not to make the switch.</p> <h3 id="portfolio"><a class="anchor-link" aria-hidden="true" href="#portfolio"></a>Portfolio</h3> <p>Thanks to features added with Wordpress 3.0, this has given me the ability to update new additions to the portfolio that much easier!</p> <h3 id="no-more-boring-fonts"><a class="anchor-link" aria-hidden="true" href="#no-more-boring-fonts"></a>No More Boring Fonts!</h3> <p>Goodbye Trebuchet MS, hello @font-face! Web fonts have wide support in a majority of browsers, so why not take advantage of it? The <a href="http://code.google.com/webfonts">Google Font API</a> is a great resource for doing so.</p> <p>This is only a portion of the site as the rest should come around the corner of the holidays. Please feel free to leave any comments or suggestions below.</p></content></entry><entry><title>Particle Generator using HTML5's Canvas</title><link href="https://scurker.com/particle-generator-using-html5s-canvas"/><id>https://scurker.com/particle-generator-using-html5s-canvas</id><updated>2010-06-04T00:00:00+00:00</updated><content type="html"><p>Particle effects are pretty awesome. Particles by themselves are fairly simple, but by generating multitudes of particles with set variables you can create a range of effects such as fire, smoke, or water. A particle generator or emitter allows you to adjust the variables giving you control over the types of effects you can generate.</p> <p>I&apos;ve been working on another project that needed a particle generator, thus this demonstration was born.</p> <p>There are several presets I&apos;ve included, but you can easily generate new types of effects by playing around with the available variables on the presets.</p> <p>The demo does not give you access to everything so in order to create more fine tuned options, here&apos;s all the currently available variables:</p> <pre><code class="language-javascript">{ <span class="attr">shape</span>: <span class="string">&apos;circle&apos;</span>, <span class="comment">// square or circle</span> velocity: <span class="keyword">new</span> Vector({<span class="attr">y</span>: <span class="number">-1</span>}), <span class="comment">// movement vector; only y is used</span> xVariance: <span class="number">0</span>, <span class="comment">// +/- start x position (random)</span> yVariance: <span class="number">0</span>, <span class="comment">// +/- start y position (random)</span> spawnSpeed: <span class="number">25</span>, <span class="comment">// # particles spawned per cycle</span> generations: <span class="number">100000</span>, <span class="comment">// # of cycles to run for</span> maxParticles: <span class="number">500</span>, <span class="comment">// max # of particles allowed on screen</span> size: <span class="number">20</span>, <span class="comment">// size of particles</span> sizeVariance: <span class="number">5</span>, <span class="comment">// +/- size of particles (random)</span> life: <span class="number">30</span>, <span class="comment">// # of cycles a particle can live</span> lifeVariance: <span class="number">10</span>, <span class="comment">// +/- lifetime of particle (random)</span> direction: <span class="number">0</span>, <span class="comment">// initial start direction</span> directionVariance: <span class="number">15</span>, <span class="comment">// +/- direction (random)</span> color: <span class="string">&apos;#fff&apos;</span>, <span class="comment">// can be hex code or rgb</span> opacity: <span class="number">1</span>, <span class="comment">// particle opacity</span> onDraw: <span class="function"><span class="keyword">function</span>(<span class="params">p</span>) </span>{ <span class="comment">// onDraw passes in the current particle and is called before each</span> <span class="comment">// particle is displayed on the screen. This function is used in</span> <span class="comment">// several of the presets to adjust the color or opacity given</span> <span class="comment">// the particle&apos;s current age and lifespan.</span> } }</code></pre> <p>If you are using Firefox and Firebug, you can create your own objects and update the particle generator by using <code>particles.update(myObject);</code> via the command line.</p> <p>As usual, HTML5 and Canvas is not currently supported by IE7/IE8, so you&apos;ll need to use another browser in order for this to work. The demo has been tested in Firefox, Safari and Chrome, but I highly recommend using Chrome for the demo as it seems to run the most efficient.</p> <p><a href="/projects/particles">On to the demo!</a> Or alternatively, view the source <a href="/projects/particles/js/particle.js">here</a>.</p></content></entry><entry><title>Javascript Clock using HTML5 and Canvas</title><link href="https://scurker.com/javascript-clock-using-html5-and-canvas"/><id>https://scurker.com/javascript-clock-using-html5-and-canvas</id><updated>2010-04-20T00:00:00+00:00</updated><content type="html"><script src="//ajax.googleapis.com/ajax/libs/jquery/1.11.1/jquery.min.js"></script> <script type="text/javascript" src="/projects/jclock/jclock.js"></script> <script type="text/javascript"> $(window).ready(function() { new jClock('/projects/jclock/clock.png', $('canvas').get(0)); }); </script> <style type="text/css"> #jclock { padding-top: 15px; padding-right: 15px; float: left; } </style> <p>I&apos;ve been finding myself becoming more interested in what HTML5 can do. As I see it HTML5 stands to be a potential replacement for flash, in addition to the features and interactivity that are made available by various javascript frameworks out there.</p> <p>A good starting point would be to start off with something simple, i.e. a clock. Here&apos;s a sample project I threw together to help make myself more familiar with HTML5&apos;s canvas.</p> <div id="jclock"> <canvas height="125" width="125"><img src="/projects/jclock/clock.png" alt="HTML5 isn&apos;t supported!" title="HTML5 isn&apos;t supported!"></canvas> </div> <p>The clock you see on the left is written completely in javascript and takes advantage of <a href="http://dev.w3.org/html5/spec/Overview.html#the-canvas-element">HTML5 and the canvas element</a> -- no flash necessary.</p> <p>If you see a blank clock face with no hands, that means that your browser does not support HTML5. You will need to be using the latest version of <a href="http://www.google.com/chrome">Chrome</a>, <a href="http://getfirefox.com">Firefox</a>, <a href="http://opera.com">Opera</a> or <a href="http://apple.com/safari">Safari</a> in order for the clock to work. IE does not currently natively support the canvas element.</p> <p>There are <a href="https://developer.mozilla.org/en/Canvas_tutorial">several</a> <a href="http://developer.apple.com/mac/library/documentation/AppleApplications/Conceptual/SafariJSProgTopics/Tasks/Canvas.html">good</a> <a href="http://dev.opera.com/articles/view/html-5-canvas-the-basics/">canvas</a> <a href="http://carsonified.com/blog/dev/html-5-dev/how-to-draw-with-html-5-canvas/">tutorials</a> out there so please check those if you want a more in-depth introduction to HTML5&apos;s canvas element.</p> <p>In order to use this clock, you&apos;ll need to setup your canvas element:</p> <pre><code class="language-html"><span class="tag">&lt;<span class="name">canvas</span> <span class="attr">id</span>=<span class="string">&quot;jclock&quot;</span> <span class="attr">height</span>=<span class="string">&quot;125&quot;</span> <span class="attr">width</span>=<span class="string">&quot;125&quot;</span>&gt;</span> Content or message to display if the browser does not support Canvas/HTML5. <span class="tag">&lt;/<span class="name">canvas</span>&gt;</span></code></pre> <p>Initializing the clock:</p> <pre><code class="language-javascript">$(<span class="built_in">window</span>).load(<span class="function"><span class="keyword">function</span>(<span class="params"></span>) </span>{ <span class="keyword">new</span> jClock(<span class="string">&apos;my-clock-face-image.png&apos;</span>, $(<span class="string">&apos;#jclock&apos;</span>).get(<span class="number">0</span>)); });</code></pre> <p>You can also pass in additional options to change items such as the hand colors, or image size. Here&apos;s the defaults as defined in the plugin:</p> <pre><code class="language-javascript"><span class="comment">// override default options:</span> <span class="comment">// i.e. new jClock(&apos;image.png&apos;, $(&apos;#canvas&apos;).get(0), {shadow: false});</span> jClock.defaults = { <span class="attr">height</span>: <span class="number">125</span>, <span class="comment">// default height</span> width: <span class="number">125</span>, <span class="comment">// default width</span> secondHand: <span class="literal">true</span>, <span class="comment">// show the second hand</span> shadow: <span class="literal">true</span>, <span class="comment">// display shadows across all hands</span> second: { <span class="comment">// second hand style options</span> color: <span class="string">&apos;#f00&apos;</span>, <span class="attr">width</span>: <span class="number">2</span>, <span class="attr">start</span>: <span class="number">-10</span>, <span class="attr">end</span>: <span class="number">35</span>, <span class="attr">alpha</span>: <span class="number">1</span> }, <span class="attr">minute</span>: { <span class="comment">// minute hand style options</span> color: <span class="string">&apos;#fff&apos;</span>, <span class="attr">width</span>: <span class="number">3</span>, <span class="attr">start</span>: <span class="number">-7</span>, <span class="attr">end</span>: <span class="number">30</span>, <span class="attr">alpha</span>: <span class="number">1</span> }, <span class="attr">hour</span>: { <span class="comment">// hour hand style options</span> color: <span class="string">&apos;#fff&apos;</span>, <span class="attr">width</span>: <span class="number">4</span>, <span class="attr">start</span>: <span class="number">-7</span>, <span class="attr">end</span>: <span class="number">20</span>, <span class="attr">alpha</span>: <span class="number">1</span> } };</code></pre> <p>Feel free to download the source code and play around with it, or ask any questions you may have in the comments below.</p> <p><a href="/projects/jclock/jclock.js">Download jclock.js (2.4k)</a></p></content></entry><entry><title>Migration from Drupal 6.x to Wordpress 2.9x</title><link href="https://scurker.com/migration-from-drupal-6-x-to-wordpress-2-9x"/><id>https://scurker.com/migration-from-drupal-6-x-to-wordpress-2-9x</id><updated>2010-02-26T00:00:00+00:00</updated><content type="html"><p>I recently found myself wanting to move my <a href="blog.scurker.com">personal blog</a> from <a href="http://drupal.org">Drupal 6.x</a> to <a href="Wordpress">Wordpress</a> for various reasons. I primarily followed <a href="http://socialcmsbuzz.com/convert-import-a-drupal-6-based-website-to-wordpress-v27-20052009">this tutorial</a>, but wanted to outline some additional information in the transfer.</p> <h3 id="database-conversion-table"><a class="anchor-link" aria-hidden="true" href="#database-conversion-table"></a>Database Conversion Table</h3> <table> <thead> <tr> <th>Drupal 6.x Table(s)</th> <th>Wordpress 2.9x Equivalent</th> </tr> </thead> <tbody> <tr> <td>term_data, term_hierarchy</td> <td>wp_terms</td> </tr> <tr> <td>node, node_revisions</td> <td>wp_posts</td> </tr> <tr> <td>term_node</td> <td>wp_term_relationships</td> </tr> <tr> <td>comments</td> <td>wp_comments</td> </tr> </tbody> </table> <h3 id="truncate-wordpress-tables"><a class="anchor-link" aria-hidden="true" href="#truncate-wordpress-tables"></a>Truncate Wordpress Tables</h3> <p>First, I needed to remove any data that is currently in certain wordpress tables so I could work with a fresh slate.</p> <p><em>Note:</em> By default when you install wordpress all tables are prefixed with <code>wp_</code> unless you changed it to something else. The below queries will need to be modified if you used anything else other than <code>wp_</code>.</p> <pre><code class="language-sql"><span class="keyword">TRUNCATE</span> <span class="keyword">TABLE</span> wp_comments; <span class="keyword">TRUNCATE</span> <span class="keyword">TABLE</span> wp_postmeta; <span class="keyword">TRUNCATE</span> <span class="keyword">TABLE</span> wp_posts; <span class="keyword">TRUNCATE</span> <span class="keyword">TABLE</span> wp_term_relationships; <span class="keyword">TRUNCATE</span> <span class="keyword">TABLE</span> wp_term_taxonomy; <span class="keyword">TRUNCATE</span> <span class="keyword">TABLE</span> wp_terms;</code></pre> <h3 id="import-taxonomy-terms"><a class="anchor-link" aria-hidden="true" href="#import-taxonomy-terms"></a>Import Taxonomy Terms</h3> <p>The next sets of queries imports taxonomy terms.</p> <p><em>Note:</em> Table names pre-pended with <code>drupal.</code> needs to be the actual name of your drupal database. You will need to change this to whatever you have your drupal database named.</p> <pre><code class="language-sql"><span class="keyword">INSERT</span> <span class="keyword">INTO</span> wp_terms (term_id, <span class="keyword">name</span>, slug, term_group) <span class="keyword">SELECT</span> d.tid, d.name, <span class="keyword">REPLACE</span>(<span class="keyword">LOWER</span>(d.name), <span class="string">&apos; &apos;</span>, <span class="string">&apos;-&apos;</span>), <span class="number">0</span> <span class="keyword">FROM</span> drupal.term_data d <span class="keyword">INNER</span> <span class="keyword">JOIN</span> drupal.term_hierarchy h <span class="keyword">USING</span>(tid);</code></pre> <p>By default, Wordpress has several taxonomy types available; <code>categories</code>, <code>post_tag</code>, and <code>link_category</code>. In my Drupal instance I used taxonomy primarily as tags, but you may have a different need. You may need to modify the 3rd line in the below query depending on how you want taxonomies imported:</p> <ul> <li><em>Categories</em>: <code>category</code></li> <li><em>Link Categories</em>: <code>link_category</code></li> <li><em>Post Tags</em>: <code>post_tag</code></li> </ul> <pre><code class="language-sql"><span class="keyword">INSERT</span> <span class="keyword">INTO</span> wp_term_taxonomy (term_taxonomy_id, term_id, taxonomy, description, <span class="keyword">parent</span>) <span class="keyword">SELECT</span> d.tid, d.tid, <span class="string">&apos;post_tag&apos;</span>, d.description, h.parent <span class="keyword">FROM</span> drupal.term_data d <span class="keyword">INNER</span> <span class="keyword">JOIN</span> drupal.term_hierarchy h <span class="keyword">USING</span>(tid);</code></pre> <h3 id="import-post-content"><a class="anchor-link" aria-hidden="true" href="#import-post-content"></a>Import Post Content</h3> <p>Drupal allows for custom post types, while as of Wordpress 2.9x, custom post types are only available via plugins. You can use the below query unmodified and it will convert all stories to posts, and everything else will transfer over as is. If you need to convert additional post types, you can add additional case statements.</p> <p>Example: <code>WHEN &apos;book&apos; THEN &apos;post&apos;</code></p> <p>I also adjusted the query so that &apos;post_date_gmt&apos; would be populated correctly based on my GMT offset of -6:00 (Central Time). If you are in a different timezone you will need to adjust <code>FROM_UNIXTIME(created+21600)</code> to subtract or add correctly based on your location.</p> <pre><code class="language-sql"><span class="keyword">INSERT</span> <span class="keyword">INTO</span> wp_posts (<span class="keyword">id</span>, post_date, post_date_gmt, post_content, post_title, post_excerpt, post_name, post_type, post_modified) <span class="keyword">SELECT</span> <span class="keyword">DISTINCT</span> n.nid, FROM_UNIXTIME(created), FROM_UNIXTIME(created+<span class="number">21600</span>), <span class="keyword">body</span>, n.title, teaser, <span class="keyword">LOWER</span>(n.title), (<span class="keyword">CASE</span> n.type <span class="keyword">WHEN</span> <span class="string">&apos;story&apos;</span> <span class="keyword">THEN</span> <span class="string">&apos;post&apos;</span> <span class="keyword">ELSE</span> n.type <span class="keyword">END</span>) <span class="keyword">as</span> <span class="keyword">type</span>, FROM_UNIXTIME(<span class="keyword">changed</span>) <span class="keyword">FROM</span> drupal.node n, drupal.node_revisions r <span class="keyword">WHERE</span> n.vid = r.vid;</code></pre> <h3 id="import-post-and-taxonomy-relationships"><a class="anchor-link" aria-hidden="true" href="#import-post-and-taxonomy-relationships"></a>Import Post and Taxonomy Relationships</h3> <pre><code class="language-sql"><span class="keyword">INSERT</span> <span class="keyword">INTO</span> wp_term_relationships (object_id, term_taxonomy_id) <span class="keyword">SELECT</span> nid, tid <span class="keyword">FROM</span> drupal.term_node;</code></pre> <h3 id="category-count-updating"><a class="anchor-link" aria-hidden="true" href="#category-count-updating"></a>Category Count Updating</h3> <pre><code class="language-sql"><span class="keyword">UPDATE</span> wp_term_taxonomy tt <span class="keyword">SET</span> <span class="keyword">count</span> = ( <span class="keyword">SELECT</span> <span class="keyword">COUNT</span>(tr.object_id) <span class="keyword">FROM</span> wp_term_relationships tr <span class="keyword">WHERE</span> tr.term_taxonomy_id = tt.term_taxonomy_id );</code></pre> <h3 id="import-comments"><a class="anchor-link" aria-hidden="true" href="#import-comments"></a>Import Comments</h3> <pre><code class="language-sql"><span class="keyword">INSERT</span> <span class="keyword">INTO</span> wp_comments (comment_post_ID, comment_date, comment_content, comment_parent, comment_author, comment_author_email, comment_author_url, comment_approved) <span class="keyword">SELECT</span> nid, FROM_UNIXTIME(<span class="built_in">timestamp</span>), <span class="keyword">comment</span>, <span class="keyword">thread</span>, <span class="keyword">name</span>, mail, homepage, <span class="keyword">status</span> <span class="keyword">FROM</span> drupal.comments;</code></pre> <h3 id="update-comment-count"><a class="anchor-link" aria-hidden="true" href="#update-comment-count"></a>Update Comment Count</h3> <pre><code class="language-sql"><span class="keyword">UPDATE</span> wp_posts <span class="keyword">SET</span> comment_count = (<span class="keyword">SELECT</span> <span class="keyword">COUNT</span>(comment_post_id) <span class="keyword">FROM</span> wp_comments <span class="keyword">WHERE</span> wp_posts.id = wp_comments.comment_post_id);</code></pre> <h3 id="update-post-slugs"><a class="anchor-link" aria-hidden="true" href="#update-post-slugs"></a>Update Post Slugs</h3> <p>Drupal&apos;s URL aliases is equivalent to Wordpress&apos; permalinks. Drupal has a much more aggressive title sanitation than Wordpress. I wanted the ability to keep my titles the same for SEO reasons when migrating over to Wordpress.</p> <p>In order to keep my old titles, I need to hook into Wordpress&apos; <a href="http://codex.wordpress.org/Function_Reference/sanitize_title">title sanitation</a> with similar rules to Drupal. The below code will need to be placed somewhere in the <code>functions.php</code> file of your current theme.</p> <pre><code class="language-php">add_filter(<span class="string">&apos;sanitize_title&apos;</span>, <span class="string">&apos;my_sanitize_title&apos;</span>); <span class="function"><span class="keyword">function</span> <span class="title">my_sanitize_title</span><span class="params">($title)</span> </span>{ $title = preg_replace(<span class="string">&apos;/\b(a|an|as|at|before|but|by|for|from|is|in|into|like|of|off|on|onto|per|since|than|the|this|that|to|up|via|with)\b/i&apos;</span>, <span class="string">&apos;&apos;</span>, $title); $title = preg_replace(<span class="string">&apos;/-+/&apos;</span>, <span class="string">&apos;-&apos;</span>, $title); $title = trim($title, <span class="string">&apos;-&apos;</span>); <span class="keyword">return</span> $title; }</code></pre> <p>You will need to save the below code to a file i.e. &quot;fix-slugs.php&quot; in your main Wordpress directory and run it through your browser.</p> <pre><code class="language-php"><span class="meta">&lt;?php</span> <span class="keyword">require_once</span>(<span class="string">&apos;wp-load.php&apos;</span>); $posts = $wpdb-&gt;get_results( <span class="string">&quot;SELECT ID, post_title, post_name FROM $wpdb-&gt;posts&quot;</span> ); $count = <span class="number">0</span>; $ignored = <span class="number">0</span>; $errors = <span class="number">0</span>; <span class="keyword">foreach</span>($posts <span class="keyword">as</span> $post) { <span class="keyword">if</span>(strcmp($slug = sanitize_title($post-&gt;post_title), $post-&gt;post_name) !== <span class="number">0</span>) { $wpdb-&gt;show_errors(); <span class="keyword">if</span>(($result = $wpdb-&gt;query(<span class="string">&quot;UPDATE $wpdb-&gt;posts SET post_name=&apos;$slug&apos; WHERE ID=$post-&gt;ID&quot;</span>)) === <span class="keyword">false</span>) { $errors++; } <span class="keyword">elseif</span>($result === <span class="number">0</span>) { $ignore++; } <span class="keyword">else</span> { $count++; } } <span class="keyword">else</span> { $ignored++; } } <span class="keyword">echo</span> <span class="string">&quot;*$count post slug(s) sanitized.*&lt;br /&gt;&quot;</span>; <span class="keyword">echo</span> <span class="string">&quot;$ignored post(s) ignored.&lt;br /&gt;&quot;</span>; <span class="keyword">echo</span> <span class="string">&quot;$errors error(s).&lt;br /&gt;&quot;</span>;</code></pre> <p>If you were following along with <a href="http://socialcmsbuzz.com/convert-import-a-drupal-6-based-website-to-wordpress-v27-20052009/">this tutorial</a>, I&apos;ve made a few changes based on my Drupal setup using <a href="http://codex.wordpress.org/Database_Description">Wordpress database description</a> as a reference when I ran into issues. There may be some additional steps to be completed if you uploaded images through Drupal&apos;s interface, but the above queries were able to successfully migrate my data from Drupal to Wordpress.</p></content></entry><entry><title>A new beginning</title><link href="https://scurker.com/a-new-beginning"/><id>https://scurker.com/a-new-beginning</id><updated>2010-01-19T00:00:00+00:00</updated><content type="html"><p>You&apos;ve undoubtedly reached this site through either word of mouth, or via my <a href="http://blog.scurker.com">personal blog</a>, but I&apos;d like to formally welcome you to the new scurker.com! I&apos;ve been hard at work over the past week in order to get everything up and running and am happy to finally be at this point.</p> <p>One of my big reasons for rolling over to a new design was the limitations my old site was giving me. I wanted to have a new fresher design in addition to having more flexibility with the content on the site. Now with the addition of <a href="http://wordpress.org">wordpress</a>, I can easily outline new projects or post about some dramatic revolution in web design. In addition the new site will help me give a better showcase for my skills and abilities for any future clients and/or employers.</p> <p>You are not seeing the final version of this roll-out, but features will slowly be added as I get time. Please feel free to post a comment if you notice any issues, or <a href="/about">contact me</a> if you have any opportunities you would like to discuss.</p></content></entry></feed>