syntax for call to java library function with byte[] (reference) parameter? - MATLAB Answers

link管理

链接快照平台

输入网页链接，自动生成快照
标签化管理网页链接

相关文章推荐

英俊的草稿纸 · boost/asio/thread_pool ...· 3 月前 ·

伤情的匕首 · 2020～2021年全国各省域政府作用竞争力 ...· 4 月前 ·

千杯不醉的电脑桌 · ELISA法检测人血清中重组人信号调节蛋白< ...· 6 月前 ·

卖萌的沙发 · RJ3150 | RuggedJet ...· 7 月前 ·

慈祥的鸵鸟 · 2024年深圳外国语学校（龙岗）国际部招生简 ...· 8 月前 ·

https://www.mathworks.com/matlabcentral/answers/66227-syntax-for-call-to-java-library-function-with-byte-reference-parameter

Sockets, again.

Where I am:

I have cast & typecast to make my data look like uint8.

I have socket & getInputStream & BufferedInputStream.

I have seen references to using java.nio.channels.Channels.newChannel(stream) instead & socket inputstreams.

I can get individual bytes from any of these with byteReturn = thing.read(). They are the uint version of my data sent. So connectivity & such are ok.

Issue:

Clearly I'd like to move to the more efficient looking

thing.read(byte[]) forms.

methodsview('java.io.InputStream') ==> read(byte[]) & read(byte[],int,int)

K>> methodsview('java.net.SocketInputStream') ==> read(... the same

      
        javaByteArray = javaArray(
       
        'java.lang.Byte'
       
        ,classStrLen);
       
        count = configS.in.read(javaByteArray);

No method 'read' with matching signature ...blah, blah

      
        javaNioByteBuffer = java.nio.ByteBuffer.allocate(1024);
       
        count = configS.channel.read(javaNioByteBuffer,0,classStrLen);

No method 'read' with matching signature found in class ...blah, blah

I've combed the Matlab help (clearly inadequately) & not found it.

Can anyone enlighten me to the syntax for a call to a java library file with a reference parameter? ie make int = read(byte[]) work or other information that will make it viable?

Or how to get data out of a java.nio.ByteBuffer if it does arrive?

I'm guessing that I can just write a java function that returns the array & call it, but it feels as if it should be possible with just a few library calls.

Thanks in anticipation.

PS: pnet, yes I know: 2 days wasted trying to get SDK to install & failed, so no compiler. (Also pnet docs seemed so much more complicated than typecast & write() --> read(byte[]) ...)

https://www.mathworks.com/matlabcentral/answers/66227-syntax-for-call-to-java-library-function-with-byte-reference-parameter#comment_161336

Bumping, since I have the same issue. Put in my words:

A Byte[] array can be created like this:

      
        target_buffer = javaArray(
       
        'java.lang.Byte'
       
        , 10);
       
        for
       
        i=1:length(target_buffer); target_buffer(i)=java.lang.Byte(0);
       
        end

Then Matlab holds a reference to the array, so if a Java function would modify it to return data I could access that from Matlab afterwards.

A byte[] array can be created, I think, like this:

target_buffer = java.util.Arrays.copyOf(int8(zeros(10,1)),10);

However, upon receiving the result Matlab converts it back to int8 instead of holding a reference. Thus it is useless to call

f = java.io.FileInputStream(filename); f.read(target_buffer, 0, 10);

because I don't have a reference to the temporary Java-buffer that was created upon sending the Matlab int8 target_buffer.

Like the original poster, I wonder if anyone is aware of a way of using methods like read(byte[]) that return by reference into a a Java primitive array? It would would be OK if it was Byte instead of byte, but for byte the autoboxing/autoconversion seems to destroy the functionality.

I found no solution in the Passing data to Java methods -documentation.

https://www.mathworks.com/matlabcentral/answers/66227-syntax-for-call-to-java-library-function-with-byte-reference-parameter#answer_288147

A SOLUTION

I haven't figured out how to pass the byte reference, but I have found a way to read a fixed number of bytes from a -Stream object. org.apache.commons.io.IOUtils comes with MATLAB (2017b, unsure when started) and toByteArray is a function that will pass the bytes directly out.

This bit of code works for me without errors.

      
        % Importing Java classes/methods to simplify code
       
        import
       
        java.io.FileInputStream
       
        import
       
        java.util.zip.GZIPInputStream
       
        import
       
        org.apache.commons.io.IOUtils.toByteArray
       
        % User settings
       
        TWO_KB              = 2048;
       
        %bytes;%"DATA_LENGTH = 2kB"
       
        GZ_PATH_CHAR        =
       
        'X:\dir\file.dat.gz'
       
        ;
       
        % Ordered cleanup tasks are necessary for more complicated operations
       
        ON_CLEANUP_TASKS    = cell(1,2)
       
        % Start file input stream
       
        FILE_INPUT_STREAM   = FileInputStream( GZ_PATH_CHAR );
       
        ON_CLEANUP_TASKS{2} = onCleanup(@()FILE_INPUT_STREAM.close);
       
        % close AFTER Gzip
       
        % Start gzip input stream
       
        GZIP_INPUT_STREAM   = GZIPInputStream( FILE_INPUT_STREAM );
       
        ON_CLEANUP_TASKS{1} = onCleanup(@()GZIP_INPUT_SRTEAM.close);
       
        % close first
       
        % read three chunks of data
       
        DATA_2KB_INT8_1 = toByteArray( GZIP_INPUT_STREAM, TWO_KB );
       
        DATA_2KB_INT8_2 = toByteArray( GZIP_INPUT_STREAM, TWO_KB );
       
        DATA_2KB_INT8_R = toByteArray( GZIP_INPUT_STREAM );
       
        % remainder of data

I've tried this with GZIPInputStream and InflaterInputStream.

https://www.mathworks.com/matlabcentral/answers/66227-syntax-for-call-to-java-library-function-with-byte-reference-parameter#comment_797235

https://www.mathworks.com/matlabcentral/answers/66227-syntax-for-call-to-java-library-function-with-byte-reference-parameter#answer_287922

Bob Gilmore, a software engineer for Mathworks at the time of this posting , indicates:

      
        [S]ince MATLAB
       
        arrays are passed to Java by value (not
       
        by
       
        reference), Java code that tries to change an input argument (like
       
        FileInputStream.read()) won't work. The only
       
        way for you to get answers
       
        back
       
        from Java is if the method you call actually *returns* a value.
       
        So
       
        I think that you have three options available:
       
        (1) Call
       
        read(no arguments) and get the data out a byte at a time
       
        , looping
       
        until
       
        you get back a -1
       
        , and
       
        stringing them together in your M-File
       
        , or
       
        (2) Write
       
        a small "pure Java" wrapper that takes the file name (or the
       
        FileInputStream) as
       
        an argument and returns the already-constructed byte
       
        array
       
        as its output.
       
        (3) Use
       
        the MATLAB file I/O methods.

He references a document I can't find on Mathworks anymore, but did find here .

      
        Since
       
        MATLAB arrays are passed by value
       
        , any
       
        changes that a Java method
       
        makes
       
        to them will not be visible to your MATLAB code. If you need to
       
        access
       
        changes that a Java method makes to an array
       
        , then, rather
       
        than
       
        passing
       
        a MATLAB array
       
        , you
       
        should create and pass a Java array
       
        , which
       
        is
       
        a
       
        reference.

https://www.mathworks.com/matlabcentral/answers/66227-syntax-for-call-to-java-library-function-with-byte-reference-parameter#comment_497612

https://www.mathworks.com/matlabcentral/answers/66227-syntax-for-call-to-java-library-function-with-byte-reference-parameter#comment_498130

https://www.mathworks.com/matlabcentral/answers/66227-syntax-for-call-to-java-library-function-with-byte-reference-parameter#answer_287736

What is a byte[] buf in MATLAB, and how do I pass it to read from the GZIP_INPUT_STREAM to retrieve a fixed number of bytes?

I'm suffering from the same problem. I have large files (+12GB over 4 GZ files) to dynamically stream in, dependent on the next header, sometimes up to 64MB chunks. I'm aware I could gunzip all the files, but this read in VERY slow.

Using the Java objects:

      
        FILE_INPUT_STREAM = javaObject(
       
        'java.io.FileInputStream'
       
        , DATA_PATH_CHAR );
       
        GZIP_INPUT_STREAM = javaObject(
       
        'java.util.zip.GZIPInputStream'
       
        , FILE_INPUT_STREAM );

I can use the standard read method one byte at a time . This is insignificantly faster than reading the files from the hard drive.

      
        DATA_BYTES_UINT8 = zeros( DATA_LENGTH, 1,
       
        'uint8'
       
        );
       
        DATA_BYTES_UINT8(:) = arrayfun( @(b)read(GZIP_INPUT_STREAM), DATA_BYTES_UINT8 );

The fastest method, however, requires the most memory, which isn't feasible for these files.

      
        BYTE_ARRAY_OUTPUT_STREAM = javaObject(
       
        'java.io.ByteArrayOutputStream'
       
        );
       
        INTERRUPTIBLE_STREAM_COPIER = com.mathworks.mlwidgets.io.InterruptibleStreamCopier.getInterruptibleStreamCopier();
       
        copyStream( INTERRUPTIBLE_STREAM_COPIER, GZIP_INPUT_STREAM, BYTE_ARRAY_OUTPUT_STREAM );
       
        DATA_BYTES_INT8 = toByteArray( BYTE_ARRAY_OUTPUT_STREAM );

The Oracle documentation describes that the read method for a GZIPInputStream accepts a byte [buffer], integer offset, and integer length of bytes to read. What is a byte[] buf in MATLAB, and how do I pass it to read from the GZIP_INPUT_STREAM to retrieve a fixed number of bytes? I'll document my failed attempts, in hopes it will draw more searches toward an eventual solution.

I tried MATLAB Java Arrays of bytes.

      
        JAVA_ARRAY = javaArray(
       
        'java.lang.Byte'
       
        , DATA_LENGTH, 1 );
       
        BYTES_READ = read( GZIP_INPUT_STREAM, JAVA_ARRAY, 0, DATA_LENGTH );
       
        % No method 'read' with matching signature found for class 'java.util.zip.GZIPInputStream'.

I've tried Java Byte Buffers.

      
        % BYTE_BUFFER = javaMethod( 'allocateDirect', 'java.nio.ByteBuffer', DATA_LENGTH ); % alt
       
        BYTE_BUFFER = javaMethod(
       
        'allocate'
       
        ,
       
        'java.nio.ByteBuffer'
       
        , DATA_LENGTH );
       
        BYTES_READ = read( GZIP_INPUT_STREAM, BYTE_BUFFER, 0, DATA_LENGTH );
       
        % No method 'read' with matching signature found for class 'java.util.zip.GZIPInputStream'.

I've tried passing an array, knowing this would not work. Oddly enough, the BYTES_READ was NOT the same as DATA_LENGTH nor was it the current position in the GZIP file. Someone correct me, but I believe the property to describe the MATLAB array would "immutable?" Is there a way to make a (u)int8 array not immutable?

      
        DATA_IN = zeros(DATA_LENGTH,1,
       
        'uint8'
       
        );
       
        BYTES_READ = read( GZIP_INPUT_STREAM, DATA_IN, 0, DATA_LENGTH );
       
        % I tried passing 0 and DATA_LENGTH as java.lang.Integer and received:
       
        % No method 'read' with matching signature found for class 'java.util.zip.GZIPInputStream'.

I've tried passing in a pointer.

      
        % DATA_PTR = libpointer( 'int8Ptr', zeros( DATA_LENGTH, 1, 'int8' ) ); % alt
       
        DATA_PTR = libpointer(
       
        'uint8Ptr'
       
        , DATA_IN );
       
        BYTES_READ = javaMethod(
       
        'read'
       
        , GZIP_INPUT_STREAM, DATA_PTR, 0, DATA_LENGTH );
       
        % No read method with appropriate signature exists in Java class java.util.zip.GZIPInputStream

I've tried another Buffered input stream, and used the interruptible stream copier. While I received ONLY the remaining data, it was not bounded to DATA_LENGTH .

      
        BUFFERED_INPUT_STREAM = javaObject(
       
        'java.io.BufferedInputStream'
       
        , GZIP_INPUT_STREAM, DATA_LENGTH );
       
        BYTE_ARRAY_OUTPUT_STREAM = javaObject(
       
        'java.io.ByteArrayOutputStream'
       
        , DATA_LENGTH );
       
        copyStream( INTERRUPTIBLE_STREAM_COPIER, BUFFERED_INPUT_STREAM, BYTE_ARRAY_OUTPUT_STREAM );
       
        DATA_IN(:) = toByteArray( BYTE_ARRAY_OUTPUT_STREAM );
       
        % In an assignment  A(:) = B, the number of elements in A and B must be the same.

Edit 1: cleared up confusing statements Edit 2: added links to GZIPInputStream Edit 3: asked if passing MATLAB array was useless because it is "immutable"

https://www.mathworks.com/matlabcentral/answers/66227-syntax-for-call-to-java-library-function-with-byte-reference-parameter#answer_416021

This is quite an old post, but I wanted to document that I found a way to pass primitive data type array references using the reflection API and Collections. It is not pretty, but can be wrapped up into a function. The example here applies to a BufferedInputStream, but the technique can easily be adapted for any general function.

EDIT: I created an FEX submission to assist with the use of this technique.

https://www.mathworks.com/matlabcentral/fileexchange/74308-javamethodwrapper

With this submission one can simply do

      
        fis = java.io.FileInputStream(filename)
       
        methodObj = JavaMethodWrapper(fis,
       
        'read(byte[], int, int)'
       
        );
       
        buf = zeros(1,1024,
       
        'int8'
       
        );
       
        [count buf] = methodObj.invoke(fis, buf, int32(0), int32(1024));

PossiblyWrong provides a similar solution as well at:

https://possiblywrong.wordpress.com/2019/07/05/calling-java-from-matlab-with-pass-by-reference-arrays/

The key is that when using invoke() in the reflection API, an Object[] is passed in with the arguments to the function. It is not possible to construct an Object[] {byte[], int, int} directly, but it IS possible to do this through ArrayList.toArray(). The following function demonstrates the technique.

      
        function
       
        out = read_BufferedInputStream(input_stream)
       
        num_available = input_stream.available();
       
        %short circuit out if none available
       
        %do not try to read, or it will block
       
        if
       
        num_available == 0
       
        out =
       
        ''
       
        ;
       
        return
       
        ;
       
        end
       
        %save the reflection method object between calls
       
        persistent
       
        m_read
       
        if
       
        isempty(m_read)
       
        %build the reflection object
       
        %we are going to lookup BufferedInputStream.read(byte[], int, int)
       
        %using reflection API
       
        getMethod_args = javaArray(
       
        'java.lang.Class'
       
        ,3);
       
        %this rather cryptic syntax is used for byte[]
       
        %since it is not possible to use java.lang.Byte[].TYPE
       
        %See:
       
        %https://docs.oracle.com/javase/specs/jvms/se7/html/jvms-4.html#jvms-4.3.2
       
        byteArrayName =
       
        '[B'
       
        ;
       
        %these are the vararg list to getMethod
       
        getMethod_args(1) = java.lang.Class.forName(byteArrayName);
       
        getMethod_args(2) = java.lang.Integer.TYPE;
       
        getMethod_args(3) = java.lang.Integer.TYPE;
       
        %use reflection to get the method object
       
        m_read = input_stream.getClass().getMethod(
       
        'read'
       
        , getMethod_args);
       
        end
       
        %save the current buffer size and array of arguments to read()
       
        persistent
       
        buf_size read_args
       
        if
       
        isempty(buf_size)
       
        MIN_SIZE = 1024;
       
        %you could set this to whatever you want
       
        %make the buffer large enough to eat all available characters in the
       
        %stream
       
        buf_size = max(MIN_SIZE, num_available);
       
        %Note that this will fail:
       
        %   read_args = javaArray('java.lang.Object', 3)
       
        %   read_args(1) = zeros(1,buf_size,'int8')
       
        %So we instead use an ArrayList which will then be converted to
       
        %Object[] for the call to invoke.
       
        read_args = java.util.ArrayList();
       
        %this will become a byte[] in the ArrayList
       
        read_args.add(zeros(1,buf_size,
       
        'int8'
       
        ));
       
        %arg for read start offset
       
        read_args.add(int32(0));
       
        %arg for read length
       
        read_args.add(int32(buf_size));
       
        end
       
        %Update the buffer to be larger if the input stream content grows
       
        if
       
        num_available > buf_size
       
        buf_size = num_available;
       
        read_args.set(0, zeros(1,buf_size,
       
        'int8'
       
        ));
       
        read_args.set(2, int32(buf_size));
       
        end
       
        %Here is the magic, when read_args is unpacked, the byte[] reference in the first
       
        %element is passed
       
        n_read = m_read.invoke(input_stream, read_args.toArray());
       
        %so now we can go back to the original ArrayList and read out the contents
       
        out = char(read_args.get(0));
       
        out = out(:)';
       
        %make row vector
       
        out = out(1:n_read);
       
        %trim to indicated size

This can be demoed with:

      
        jf = java.io.File(
       
        'path\to\myfile.txt'
       
        );
       
        fis = java.io.FileInputStream(jf);
       
        bis = java.io.BufferedInputStream(fis);
       
        my_file_text = read_BufferedInputStream(bis);

https://www.mathworks.com/matlabcentral/answers/66227-syntax-for-call-to-java-library-function-with-byte-reference-parameter#comment_838224

Well done Benjamin Davis! Your discovery of java.util.ArrayList 's toArray() has opened various Java-usage doors to us! I post below an example function that converts MATLAB's variables to Java objects and classes and classnames.

      
        % This uses java.util.ArrayList to convert varargin to (1) java.lang.Object
       
        % array, and (2) java.lang.Class array. Output (1) can be used i.e. as
       
        % second input to the Java method's invoke-call. It is noteworthy that this
       
        % way any modifications to its arrays, i.e. primitive type arrays, can be
       
        % collected and used at the MATLAB side after the invoke-call! Output (2)
       
        % can be used i.e. to find a method with the matching signature! Output (3)
       
        % can be used i.e. to quickly inspect what class types Java side has
       
        % obtained in human-readable form. Read the reference [1] to understand how
       
        % MATLAB types are mapped to Java types. Idea for this code was inspired by
       
        % the reference [2].
       
        % [1] https://www.mathworks.com/help/matlab/matlab_external/passing-data-to-java-methods.html
       
        % [2] https://www.mathworks.com/matlabcentral/answers/66227-syntax-for-call-to-java-library-function-with-byte-reference-parameter#answer_416021
       
        function
       
        [jObjects, jClasses, classnames] = java_objects_from_varargin(varargin),
       
        % Convert varargin to a java.lang.Object array via java.util.ArrayList
       
        jArrayList = java.util.ArrayList();
       
        for
       
        ii = 1:nargin,
       
        jArrayList.add(varargin{ii});
       
        % Benefit from Java's autoboxing feature!
       
        end
       
        jObjects = jArrayList.toArray();
       
        % Get constructed java.lang.Object array
       
        % Exit here if second or higher outputs are not used
       
        if
       
        nargout < 2,
       
        return
       
        ;
       
        end
       
        % Get varargin classes by reflecting getClass-method of java.lang.Class
       
        % in order to force java.lang.Object. Direct call to getClass will fail
       
        % for primitive types.
       
        jC_Class = java.lang.Class.forName(
       
        'java.lang.Class'
       
        );
       
        jM_getClass = jC_Class.getMethod(
       
        'getClass'
       
        , []);
       
        jClasses = java.lang.reflect.Array.newInstance(jC_Class, nargin);
       
        % Backward-compatible, because javaArray cannot initialize zero length array in older MATLAB versions.
       
        for
       
        ii = 1:nargin,
       
        if
       
        ~isempty(jObjects(ii)),
       
        % Continue if not null
       
        % Although SCALAR primitive types (byte, short, int, long,
       
        % float, double, boolean, char in Java) are correctly autoboxed
       
        % by their corresponding Java object wrappers on the creation
       
        % of java.util.ArrayList, their classes cannot be reliably
       
        % obtained due subsequent unboxing/autoboxing and MATLAB
       
        % interference. Their classes must be manually set.
       
        if
       
        numel(varargin{ii}) == 1,
       
        % Continue only if scalar value
       
        matlab_class = {
       
        'logical'
       
        ,
       
        'char'
       
        ,
       
        'int8'
       
        ,
       
        'uint8'
       
        ,
       
        'int16'
       
        ,
       
        'uint16'
       
        ,
       
        'int32'
       
        ,
       
        'uint32'
       
        ,
       
        'int64'
       
        ,
       
        'uint64'
       
        ,
       
        'single'
       
        ,
       
        'double'
       
        };
       
        java_class = {
       
        'Boolean'
       
        ,
       
        'Character'
       
        ,
       
        'Byte'
       
        ,
       
        'Byte'
       
        ,
       
        'Short'
       
        ,
       
        'Short'
       
        ,
       
        'Integer'
       
        ,
       
        'Integer'
       
        ,
       
        'Long'
       
        ,
       
        'Long'
       
        ,
       
        'Float'
       
        ,
       
        'Double'
       
        };
       
        B_match = strcmp(matlab_class, class(varargin{ii}));
       
        if
       
        any(B_match),
       
        java_wrapper_classname = [
       
        'java.lang.'
       
        java_class{B_match}];
       
        jClasses(ii) = eval([java_wrapper_classname
       
        '.TYPE'
       
        ]);
       
        % Call wrapper's static TYPE field
       
        continue
       
        ;
       
        % Done
       
        end
       
        end
       
        % Arrays of primitive types require indirect call to getClass()
       
        jClasses(ii) = jM_getClass.invoke(jObjects(ii), []);
       
        % Subsequent unboxing/autoboxing event
       
        end
       
        end
       
        % Exit here if third or higher outputs are not used
       
        if
       
        nargout < 3,
       
        return
       
        ;
       
        end
       
        classnames = cell(nargin, 1);
       
        for
       
        ii = 1:nargin,
       
        if
       
        ~isempty(jClasses(ii)),
       
        % Continue if not null
       
        classnames{ii} = char(jClasses(ii).getName());
       
        end
       
        end
       
        end

Other advanced Java functions I have developed can be found here . I am hoping that they will inspire all us to benefit more from Java + MATLAB combo.

One more thing:

In my decompression code, I have used jByteBuffer = java.nio.ByteBuffer.allocate(int); in combination with jChannel = java.nio.channels.Channels.newChannel(InputStream); to replace any N_read = jStream.read(byte [], int, int); with N_read = jChannel.read(ByteBuffer); . This ByteBuffer object can then be converted back to bytes by buffer = jByteBuffer.array(); . Its main benefit is that I do not need to use these ArrayList and Java reflection tricks, which may also have performance penalty, but I can instead directly call read of java.nio package, which has existed since Java 4! For instance, R2011a uses Java 6 and R2019b uses Java 8.

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!