Java 8: Streaming a string

It's time to finally start learning java 8 api. Recently I needed to modify a string by replacing some characters to randomly generated values. One method for '#' -> random number, one method for '?' -> random lower case letter and one for both replacements. Each char should be processed independently, therefore simply replaceAll was not an option. However mapping a stream sounds easy. Unfortunately java doesn't provide CharStream and CharJoiner so it's a bit uglier and less efficient than it should be.

import static org.apache.commons.lang3.RandomStringUtils.randomAlphabetic;
import static org.apache.commons.lang3.RandomStringUtils.randomNumeric;
import java.util.function.Supplier;
import java.util.stream.Collectors;
import com.google.common.base.Preconditions;

public class Replacer {

  private String replace(String toReplace, char pattern, Supplier<String> replacementSupplier) {
    Preconditions.checkNotNull(toReplace);
    return toReplace
             .chars()
             .mapToObj(c -> c != pattern ? String.valueOf((char)c) : replacementSupplier.get())
             .collect(Collectors.joining());
  }
 
  public String withLetters(String letterString) {
    return replace(letterString, '?', () -> randomAlphabetic(1).toLowerCase());
  }

  public String withNumbers(String numberString) {
    return replace(numberString, '#', () -> randomNumeric(1));
  }

  public String withBoth(String string) {
    return withNumbers(withLetters(string));
  }
}

I know this can be done more efficiently using direct operations on ints. But can this be done more efficiently without sacrificing readability? Any suggestions are welcome.

Make git never asks for your github username again

A lot of answers in SO advise you to change your repo url after you cloned the project. Don't do it. Don't change your repo url per project. That's a waste of time. Soon you will clone another project and you will have to do it again. Better way is to set the username globally so it applies to all projects from github. Current and future ones. Just type:

git config --global credential."https://github.com".username YOUR_USERNAME

This will add

[credential "https://github.com"]
username = YOUR_USERNAME

to your ~/.gitconfig

Nesting angular directives

Using angular you quickly find directives really handy. Often you can google a directive that does exactly what you need... almost. For example there is plenty of ready-to-use multichoice dropdowns. But let's say we need a dropdown that is automatically disabled when there is no possible choices. One way is to get the source and change it - with angular it's usually easy. But then we are stuck with this version and won't get any updates nor bugfixes. Better solution is to wrap an existing directive in our own.

Let's say we have an existing dropdown-choices directive that uses one parameter (dropdown-choices) for providing all possible choices and another one (dropdown-selected) for currently selected value. Let's use it to get what we need.

Tests first. Make sure your karma.conf.js contains a reference to dropdown-choices, for example:

module.exports = function(config) {
  config.set({
    ...
    files: [
      ...
      'app/vendor/*.js',
      ...

Example tests

'use strict';

describe('autoOffDropdown', function() {

    var autoOffDropdownDiv, scope;

    beforeEach(module('myModule'));
    beforeEach(inject(function($rootScope, $compile) {
        scope = $rootScope.$new();
        multicomboDiv = $compile('<div id="someId" auto-off-dropdown  \
                                       choices="choices" selected="selected"></div>')(scope);
    }));

    it('should be disabled when there is nothing to select', function() {
        scope.choices = [];

        expectSingleVisibleChildToBe("input:disabled");
    });

    it('should show div with dropdown when anything can be selected', function() {
       scope.choices = [1];

       expectSingleVisibleChildToBe("div");
    });

    function expectSingleVisibleChildToBe(jquerySelector) {

       scope.$apply();
       // may differ depending on angular version
       var visibleChildren = multicomboDiv.children().filter(
                                    function() {return $(this).css('display') !== 'none';});

       expect(visibleChildren.length).toBe(1);
       expect(visibleChildren.is(jquerySelector)).toBe(true);
   }
});

And now our directive

'use strict';

angular.module("myModule").directive('autoOffDropdown',function(){

    return {
        restrict: 'A',
        replace: true,
        scope: {
            choices:  '=',
            selected: '='
        },
        template:
            '
\
                
\
                    
\
                
\
                <input data-ng-show="noChoicesAvailable()" type="text" disabled="disabled" \
                                                           placeholder="No choices available"/> \
             
',
        link: function(scope) {
            scope.noChoicesAvailable = function() {
                return _.isEmpty(scope.choices);
            };
        }
    };
});

Tested with angular 1.0.8

Mocking $location in angular tests

Let's say we want to mock a single service, i.e. $location. In case of internal angular's services we can use dedicated mocks, like the one described here, but often something much simpler is enough. Let's say we have a service:

angular.module('myModule').factory('myService', function($location) {
   return {
      urlSize: $location.absUrl().length
   }
});

and now we want to test it. Obviously all we need is just an object with a single function. In such case, for this test we can simply provide a new definition of the required service:

var url;

module('myModule');
module(function($provide) {
   $provide.factory('$location', function() {
      return {
         absUrl: function () {return url}
      }
   })
});

and now our new $location is registered in angular's DI container and will be provided to all dependant services. Furthermore this test is a clear documentation that myService uses only this one single function of $location. There are no spies that obfuscate behaviour of tested code. How can we use this mock? It's a closure so in every test we can change the global variable url and the mocked $location.absUrl() will return this value. Now we can simply inject the service we want to test:

inject(function(myService) {
   expect(myService.urlSize).toEqual(...);
});

Tested with angular 1.0.8 and 1.2.16

9 lines that made me learn monads

Some time ago I saw this programing languages comparison based on a small task. The task is to implement evaluation system for simple expressions: addition and multiplication of numbers and variables. Most solutions (regardless of the language) look like the first one:

(use '[clojure.core.match :only [match]])
 
(defn evaluate [env [sym x y]]
  (match [sym]
    ['Number]   x
    ['Add]      (+ (evaluate env x) (evaluate env y))
    ['Multiply] (* (evaluate env x) (evaluate env y))
    ['Variable] (env x)))

(def environment {"a" 3, "b" 4, "c" 5})
 
(def expression-tree '(Add (Variable "a") (Multiply (Number 2) (Variable "b"))))
 
(def result (evaluate environment expression-tree)) 

It boils down to defining 4 types of expressions, each of which receives subexpressions and the environment and later passes the environment to the subexpressions. Easy, right? There are also a few solutions that avoid passing the environment. Instead they have 'evaluate' function that preprocesses the expression tree and later evaluates it without the environment at all. For example they replace variables with their values upfront.

And then I saw this:

import Data.Map
import Control.Monad

number   = return
add      = liftM2 (+)
multiply = liftM2 (*)
variable = findWithDefault 0

environment = fromList [("a",3), ("b",4), ("c",7)]

expressionTree = add (variable "a") (multiply (number 2) (variable "b"))

result = expressionTree environment

And I was wondering how does it work. Where the hell is environment passing or 'evaluate' function?! Can you write such code? If not, it's time to learn monads.

Spring boot: @DependsOn is not enough anymore

Spring boot can really help speed up starting new project. Ten lines of java configuration (mostly generated by let's say data-jpa-mvn archetype) and you are ready to write your business logic. All the configuration changes can be postponed until you really need them. But when you start doing those changes, very quickly you may see that something is reeeeeally missing.

Let's say we want to use beans A and C created by spring boot. But also we want to create our own bean B that should be initialized in between. The real word scenario is for example: default DataSource, default EntityManagerFactory and customized Flyway service which must run before hibernate. We can easily say flyway should depend on the datasource - @Autowired does the trick. But how can we say that hibernate should depend on our flyway? We can't add anything to the hibernate bean because we don't declare it. So what's the solution? First let's check how and where spring declares the hibernate. After listing *AutoConfiguration classes we see HibernateJpaAutoConfiguration and later, in its superclass, we can find:

@Bean
@ConditionalOnMissingBean(name = "entityManagerFactory")
public LocalContainerEntityManagerFactoryBean entityManagerFactory(
                           JpaVendorAdapter jpaVendorAdapter) {...}

So one way of enforcing the order is:

@Configuration
class FlywayConfig extends HibernateJpaAutoConfiguration {

   @Autowired Flyway flyway;
}

And if we need more precise control, we can override the bean:

@Configuration
class FlywayConfig extends HibernateJpaAutoConfiguration {
   @Bean
   public LocalContainerEntityManagerFactoryBean entityManagerFactory(
                                   JpaVendorAdapter jpaVendorAdapter,
                                   Flyway flyway) {
      return super.entityManagerFactory(jpaVendorAdapter);
   }
}

Used versions: spring 4.0.3.RELEASE, spring-boot 1.0.2.RELEASE

Encrypted filesystem on top of an LVM

I always forget how to create encrypted partition inside an lvm, so that's a good reason to write it. I'm not an admin so there might be better ways to do the same.

Objective: To have an encrypted device, unmounted by default, ready for one-click mount that asks for the password.

My tools:

• lvm2
• system-config-lvm - gui for LVM management
• palimpset - gui that allows to mount, unmount encrypted LUKS devices inside lvm volumes, renaming VG (volume group), LV (logical volume) and filesystem inside encrypted LUKS volume
• ubuntu 12.04 LTS
• mate (it offers one-click mount but probably any other desktop environment will do)

During the whole process tab button will be your friend. It will help with luks subcommands, with finding luks devices etc. So let's start:

Creating encrypted FS

1. Create partitions that don't waste space but can store your extents and lvm metadata. Something around (n*extent_size)+1mb. Create new partition in Gparted, format as LVM and compare the total size with the available size.
2. Create desired VGs (my_VG) and LVs (my_LV) using system-config-lvm
3. Create luks volume using whole available space on created LV. You will be asked for a passphrase
   

   sudo cryptsetup luksFormat /dev/mapper/my_VG-my_LV
   

4. Open created luks volume and register it under custom name (my_luks)
   

   sudo cryptsetup luksOpen /dev/mapper/my_VG-my_LV my_luks

5. Create filesystem using whole available space on opened luks volume
   

   mkfs.ext4 /dev/mapper/my_luks
   

6. (Un)mount volumes and change all the labels using sudo palimpset

That's it. Your system (places or drivemount_applet2) will report existing encrypted LV waiting to be mounted. By default file system label will be used as a mount point for on-click mount.

Shrinking encrypted FS

There is often some confusion which tools use kilo- prefix for 2^10 and which for 10^3. It looks like all commands used below use the 2^10.

1. Prepare for the shrinking. Open luks, check the FS size and the FS itself.
   

   sudo cryptsetup luksOpen /dev/mapper/my_VG-my_LV my_luks
   sudo mount -r /dev/mapper/my_luks /any_custom_dir/
   df -h
   df -B1k   # show size in kilobytes (2^10 bytes)
   sudo umount /any_custom_dir/
   e2fsck -f /dev/mapper/my_luks
   

2. Shrink the FS a bit more than you really need to (about 90%). Provide new size or shrink it maximally with -M. Option -p adds a progress bar. In my case (ext4) system didn't allow me to shrink below the possible minimum size so it looks like you don't have to calculate the new size very precisely. Units: 2^10
   

   resize2fs -Mp /dev/mapper/my_luks

   or
   

   resize2fs -p /dev/mapper/my_luks 50G

   Check if all is good
   

   e2fsck /dev/mapper/my_luks

3. Shrink the LV. there is no need to shrink the luks volume as it doesn't have the concept of 'fixed size'. During every mount, luks uses all the space of the underlying device. But shrinking the LV is the tricky part. I couldn't find a way to precisely calculate the space. It looks like df shows FS size including its metadata but I don't know which size is used by resize2fs. Anyway, remember about the FS metadata (a few percent) and luks metadata (around 2MB). So add about 10% (depending on your FS type) to the FS size and shrink LV to that size.
   
   There are many ways to expres the new size (absolute value, difference, percentage etc. use man). The man never says if it uses 2^10 or 10^3 units but it 'seems' like the first one.
   
   LVM will not warn or prevent you from overwriting your data so bad assumption, calculation or a typo will result in a data loss.
   

   sudo cryptsetup luksClose my_luks
   sudo lvreduce -L 900G /dev/my_VG/my_LV
   

   Check how big damages were made
   

   sudo cryptsetup luksOpen /dev/mapper/my_VG-my_LV my_luks
   e2fsck -f /dev/mapper/my_luks
   

4. If e2fsck said that you've just shrunk your LV too much, don't panic. Just close luks, extend LV a bit lvextend -L, open luks and check your FS again.
   df said my FS was 811G but shrinking LV to 820G (lvreduce -L 820G) was too much. However extending it to 830G was enough.
   


5. Grow the FS to fill whole underlying LV, otherwise it would be a waste of space
   

   resize2fs -p /dev/mapper/my_luks
   e2fsck /dev/mapper/my_luks
   

That's all. Some other useful commands to check the status of devices: http://ubuntuforums.org/showthread.php?t=726724