The
memcached
open-source library is great for sending objects from process to process. It is typically used to cache large data objects from databases or in a distributed environment, but we can also use it as a simple distributed shared memory system, which stores data in key-value pairs (as in a hashtable).
I was able to use memcached to set and retrieve objects from two different instances of Matlab. Both the Java and .NET variants of memcached can be used for this. memcached requires two components: server and client: The server comes precompiled, while the client needs to be compiled by us.
The memcached server was compiled by Trond Norbye from Couchbase (previously Membase and Northscale) for
32-bit
and
64-bit
Windows OS. See
below
how to run it as a service using the Python PyWin32 package.
I compiled Windows binaries of memcached-client libraries for both the Java and .NET using Java-1.7 and Visual Studio 2010 and zipped them up
here
.
Note: additional (non-memcached) shared-memory implementations used in Matlab include Joshua Dillon’s
sharedmatrix
(also see
here
), and Kevin Stone’s
SharedMemory
utilities, which use POSIX shared-memory and the
Boost IPC library
. Rice University’s
TreadMarks
library is another example of a shared-memory approach that has been used with Matlab, in the
MATmarks
package (note that the current availability of MATmarks is unclear).
We use the
Memcached server version 1.4.5 patched for 64-bit Windows
, precompiled by Trond Norbye from Couchbase (previously Membase and Northscale).
For the Memcached .NET client, download the
Memcached.ClientLibrary
port of
com.meetup.memcached
from SourceForge, and then compile using Microsoft Visual Studio 2010 C#, Release/AnyCPU configuration. Since the original project files were for MSVC-8, MSVC-10 starts a conversion wizard that first backed up the previous project files and then creates a new MSVC-10 project, including a new solution file. I then build the
clientlib_2.0
project from the MSVC-10 IDE GUI using “build” from the menu, and voila: I got a new DLL.
Usage in Matlab is as follows:
Start the memcached server:
C:\> C:\path\to\memcached.exe -vv
C:\> C:\path\to\memcached.exe -vv
Do this on both MATLAB instances:
asm = NET.addAssembly('C:\full\path\to\Memcached.ClientLibrary.dll');
pool = Memcached.ClientLibrary.SockIOPool.GetInstance();
pool.SetServers({'127.0.0.1:11211'});
pool.Initialize;
mc = Memcached.ClientLibrary.MemcachedClient();
asm = NET.addAssembly('C:\full\path\to\Memcached.ClientLibrary.dll');
pool = Memcached.ClientLibrary.SockIOPool.GetInstance();
pool.SetServers({'127.0.0.1:11211'});
pool.Initialize;
mc = Memcached.ClientLibrary.MemcachedClient();
On MATLAB instance #1:
mc.Set('name','mark')
mc.Set('name','mark')
On MATLAB instance #2:
>> myName = mc.Get('name')
myName =
mark
>> myName = mc.Get('name')
myName =
MATLAB and memcached also works perfectly with the original meetup.com Java version that the .NET version was ported from. The commands are exactly the same, but the Apache logger is not cleaned up for the non-bench/test sources (with
isErrorEnabled()
, etc. see
jlog4
documentaion
), so you always get this error message:
log4j:WARN No appenders could be found for logger (com.meetup.memcached.SockIOPool).
log4j:WARN Please initialize the log4j system properly.
Because there are log calls in the MemcachedClient and SockIOPool files that never have
BasicConfiguration.configure()
or set any
jlog4.
like the
appender
hence the error. Of course they never meant for log messages to be displayed when deployed, so wrapping with
isErrorEnabled()
like the .NET version does would be better.
The Java package is called
com.meetup.memcached
. We should build it ourselves from the
Github source
. The jar from the
Maven repository
didn’t work (attempting to start the sock I/O pool raised several Java errors due to missing includes), and differs a bit from the GitHub repo.
To use in Matlab, first, start the memcached server, then start sock I/O pools and memcached clients on both Matlab instances:
>> javaaddpath('C:\full\path\to\MeetupMemcached.jar');
>> pool = com.meetup.memcached.SockIOPool.getInstance();
>> pool.setServers({'127.0.0.1:11211'})
;
>> pool.initialize;
>> mc = com.meetup.memcached.MemcachedClient;
>> mc.set('name','mark'); % on 1st instance/process
>> myName = mc.get('name')% on 2nd instance/process
myName =
mark
>> javaaddpath('C:\full\path\to\MeetupMemcached.jar');
>> pool = com.meetup.memcached.SockIOPool.getInstance();
>> pool.setServers({'127.0.0.1:11211'});
>> pool.initialize;
>> mc = com.meetup.memcached.MemcachedClient;
>> mc.set('name','mark'); % on 1st instance/process
>> myName = mc.get('name') % on 2nd instance/process
myName =
Other clients
xmemcached
– Github:killme2008, Google Code project and Maven repo
The Xmemcached client works well. It is the most recently updated. There are jar files on the releases page of the GitHub repo. Here is an example from the 2.0.0 release from this April. The example is from the google.code wiki User Guide in English.
>> javaaddpath('C:\full\path\to\xmemcached-2.0.0.jar');
>> addr = net.rubyeye.xmemcached.utils.AddrUtil.getAddresses('localhost:11211')
addr =
[localhost/127.0.0.1:11211]
>> builder = net.rubyeye.xmemcached.XMemcachedClientBuilder(addr)
builder =
net.rubyeye.xmemcached.XMemcachedClientBuilder@7ef6a26
>> mc = builder.build()
log4j:WARN No appenders could be found for logger (net.rubyeye.xmemcached.XMemcachedClient).
log4j:WARN Please initialize the log4j system properly.
net.rubyeye.xmemcached.XMemcachedClient@275e6ce5
>> mc.set('name',0,'mark')ans =
>> myName = mc.get('name')
myName =
mark
>> javaaddpath('C:\full\path\to\xmemcached-2.0.0.jar');
>> addr = net.rubyeye.xmemcached.utils.AddrUtil.getAddresses('localhost:11211')
addr =
[localhost/127.0.0.1:11211]
>> builder = net.rubyeye.xmemcached.XMemcachedClientBuilder(addr)
builder =
net.rubyeye.xmemcached.XMemcachedClientBuilder@7ef6a26
>> mc = builder.build()
log4j:WARN No appenders could be found for logger (net.rubyeye.xmemcached.XMemcachedClient).
log4j:WARN Please initialize the log4j system properly.
net.rubyeye.xmemcached.XMemcachedClient@275e6ce5
>> mc.set('name',0,'mark')
ans =
>> myName = mc.get('name')
myName =
Enyim
– NuGet and Github:enyim
I couldn’t get this assembly to load as built, I always got an “Strong Name Validation Failed” error, perhaps because I am on a 64-bit machine, but using MSVC express.
spymemcached
– Github:dustin, Google Code project and Maven repo
I also couldn’t get this to work. Even though I could make an array of Java socket objects using
InetSocket
the connection was always refused even though the memcached server was operating on the specified port.
memcached server as Windows service
This
Gist
uses Python PyWin32
win32service
to install, remove, configure, start and stop the memcached server.
Related posts:
Serializing/deserializing Matlab data
–
Matlab does not provide a documented manner to serialize data into a byte stream, but we can do this with some undocumented functionality. ...
Simulink Data Dictionary
–
Simulink contains undocumented public API for access to its data dictionary functionality. ...
Accessing plot brushed data
–
Plot data brushing can be accessed programmatically using very simple pure-Matlab code...
Sparse data math info
–
Matlab contains multiple libraries for handling sparse data. These can report very detailed internal info. ...
Additional license data
–
Additional meta-data about installed toolboxes can be retrieved in Matlab. ...
Controlling plot data-tips
–
Data-tips are an extremely useful plotting tool that can easily be controlled programmatically....
In order to get rid of that message and output logging messages to a file configure the Apache.org log4j logger yourself by creating a
FileAppender
with the default layout and then configuring the
BasicConfigurator
.
To get rid of the logger message for xmemcached you will have to download
SLF4J
and add
slf4j-api-X.Y.Z.jar
,
slf4j-simple-X.Y.Z.jar
and
slf4j-log4j-X.Y.Z.jar
to your path first, where X.Y.Z is the SLF4J version, 1.7.7 as of 2014-08-27. Then configure log4j as above for com.meetup.memcached.
Turns out that the newer schooner/whalin client is even faster than the old whalin (meetup.com) client, and they say it’s faster than both spymemcached (from couchbase/membase) and xmemcached.
The missing libraries to make this run are in the lib folder of the
zip file for the Memcached-Java-Client-3.0.0
which are the release page of Greg Whalin’s Github Memcached-Java-Client repo.
I was able to build this using jdk-7 by extracting the file, making a new
bin
folder at the same folder as
lib
and
src
, then navigating to
src\main\java
and executing the following:
It works almost exactly like the old meetup.com client, but the package is named “com.whalin.MemCached” instead. Also now you will also have to add “commons-pool-1.5.6.jar”, “slf4j-api-1.6.1.jar” and “slf4j-simple-1.6.1.jar” to your MATLAB Java classpath using
javaaddpath()
.
HTML tags such as <b> or <i> are accepted.
Wrap code fragments inside <pre lang="matlab"> tags, like this:
<pre lang="matlab"> a = magic(3); disp(sum(a)) </pre>
I reserve the right to edit/delete comments (read the site
policies
).
Not all comments will be answered. You can always email me (altmany at gmail) for private
consulting
.
Leuze (20 days 21 hours ago)
: Hello, I would like to add a new object into SLDD and display columns according to my object properties <>
Eric Delgado (50 days 0 hours ago)
: Hey Yair, I think your site is the right place to share that I wrote ccTools, a lib that allows a lot of customizations of Matlab built-in web components (such as uifigure,...
Eric Delgado (50 days 0 hours ago)
: Hey guys, first of all, thanks to @Yair, your site saved me a lot of times! 🙂 That’s why I am sharing that I wrote m2mlapp (*.m => *.mlapp) file conversion, that...
Yair Altman (67 days 20 hours ago)
: I am not familiar with a universal catch-all tab-completion mechanism for all functions, and it also doesn’t make much sense to me, because different functions use...
Arye (68 days 14 hours ago)
: hi, is there a universal functionSignature.json file that will allow my to get the function hints when opening brackets of my funcs? something that automaticly recognize my function...
Sunham (71 days 8 hours ago)
: This is an old article, but the issue persists even in 2023. 2023a: z = mat2cell(1:1e6,1,repmat(1,1,1e 6)); f = @() cellfun(‘isempty’, z); g = @() cellfun(@isempty,z);...
Yair Altman (80 days 23 hours ago)
: Robot only runs when you tell it to run a command such as keyPress. If you don’t tell it to run a command, it uses no CPU, so there’s no need to remove the Robot...
Eric (81 days 10 hours ago)
: Hey @Kevin, can you share your code about create group of figures in the AppContainer? The container of multiples uifigures could be an amazing improvement over AppDesigner and its...
Elsa Smith (82 days 1 hour ago)
: I recently used java.awt.Robot to perform GUI testing on MATLAB and found it to be an extremely easy and useful way to control mouse movements.
Elsa Smith (82 days 1 hour ago)
: I’m suspecting that the slow performance of my GUI may be due to the use of java.awt.Robot. Is there a way to cancel/stop/remove the robot after it has been created, or is...
Michelle Kline (82 days 18 hours ago)
: Thank you, Yair! With this previously-unknown-to-me tip about fwrite() performance, you have saved me literally hours of processing time. Michelle Kline Department of...
Alessandro Beda (95 days 6 hours ago)
: I found what I think is a bug related to this (tested in R2022 and R2023a). If I add a “ButtonDownFcn” to the plots (see example below), then the modified...
Nicholas (96 days 20 hours ago)
: Yair, Changing the desktop help options did not solve the issue. Though, it’s unclear how I could change these options in the Runtime, if that’s what you meant? I should...
Recent Comments