1/* Part of SWI-Prolog 2 3 Author: Jeffrey Rosenwald, extended by Peter Ludemann 4 E-mail: jeffrose@acm.org, peter.ludemann@gmail.com 5 WWW: http://www.swi-prolog.org 6 Copyright (c) 2010-2013, Jeffrey Rosenwald 7 All rights reserved. 8 9 Redistribution and use in source and binary forms, with or without 10 modification, are permitted provided that the following conditions 11 are met: 12 13 1. Redistributions of source code must retain the above copyright 14 notice, this list of conditions and the following disclaimer. 15 16 2. Redistributions in binary form must reproduce the above copyright 17 notice, this list of conditions and the following disclaimer in 18 the documentation and/or other materials provided with the 19 distribution. 20 21 THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS 22 "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT 23 LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS 24 FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE 25 COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, 26 INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, 27 BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; 28 LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER 29 CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT 30 LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN 31 ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE 32 POSSIBILITY OF SUCH DAMAGE. 33*/ 34 35:- module(protobufs, 36 [ protobuf_message/2, % ?Template ?Codes 37 protobuf_message/3, % ?Template ?Codes ?Rest 38 protobuf_parse_from_codes/3, % +WireCodes, +MessageType, -Term 39 protobuf_serialize_to_codes/3, % +Term, +MessageType, -WireCodes 40 protobuf_field_is_map/2, % +MessageType, +FieldName 41 protobuf_map_pairs/3 % ?ProtobufTermList, ?DictTag, ?Pairs 42 43 % TODO: Restore the following to the public interface, if 44 % someone needs them. For now, the tests directly specify 45 % them using, e.g. protobufs:uint32_codes(..., ...). 46 % 47 % protobuf_segment_message/2, % ?Segments ?Codes 48 % protobuf_segment_convert/2, % +Form1 ?Form2 49 % uint32_codes/2, 50 % int32_codes/2, 51 % float32_codes/2, 52 % uint64_codes/2, 53 % int64_codes/2, 54 % float64_codes/2, 55 % int64_zigzag/2, 56 % uint32_int32/2, 57 % uint64_int64/2, 58 % uint32_codes_when/2, 59 % int32_codes_when/2, % TODO: unused 60 % float32_codes_when/2, 61 % uint64_codes_when/2, 62 % int64_codes_when/2, % TODO: unused 63 % float64_codes_when/2, 64 % int64_zigzag_when/2, 65 % uint32_int32_when/2, 66 % uint64_int64_when/2, 67 % int64_float64_when/2, 68 % int32_float32_when/2, 69 % protobuf_var_int//1, 70 % protobuf_tag_type//2 71 ]). 72 73:- use_module(library(apply_macros)). % autoload(library(apply), [maplist/3, foldl/4]). 74:- autoload(library(error), [must_be/2, domain_error/2, existence_error/2]). 75:- autoload(library(lists), [append/3]). 76:- autoload(library(utf8), [utf8_codes//1]). 77:- autoload(library(dif), [dif/2]). 78:- autoload(library(dcg/high_order), [sequence//2]). 79:- autoload(library(when), [when/2]). 80:- autoload(library(debug), [assertion/1]). % TODO: remove 81 82:- set_prolog_flag(optimise, true). % For arithmetic using is/2.
162:- use_foreign_library(foreign(protobufs)).MessageType), creating a Prolog term.
Protoc must have been run (with the --swipl_out= option and the resulting
top-level _pb.pl file loaded. For more details, see the "protoc" section of the
overview documentation.
Fails if the message can't be parsed or if the appropriate meta-data from protoc
hasn't been loaded.
All fields that are omitted from the WireCodes are set to their
default values (typically the empty string or 0, depending on the
type; or [] for repeated groups). There is no way of testing
whether a value was specified in WireCodes or given its default
value (that is, there is no equivalent of the Python
implementation's =HasField`). Optional embedded messages and groups
do not have any default value -- you must check their existence by
using get_dict/3 or similar. If a field is part of a "oneof" set,
then none of the other fields is set. You can determine which field
had a value by using get_dict/3.
219protobuf_parse_from_codes(WireCodes, MessageType0, Term) :- 220 verify_version, 221 must_be(ground, MessageType0), 222 ( proto_meta_normalize(MessageType0, MessageType) 223 -> true 224 ; existence_error(protobuf_package, MessageType0) 225 ), 226 protobuf_segment_message(Segments, WireCodes), 227 % protobuf_segment_message/2 can leave choicepoints, backtracking 228 % through all the possibilities would have combinatoric explosion; 229 % instead use segment_to_term/3 call protobuf_segment_convert/2 to 230 % change segments that were guessed incorrectly. 231 !, 232 maplist(segment_to_term(MessageType), Segments, MsgFields), 233 !, % TODO: remove 234 combine_fields(MsgFields, MessageType{}, Term), 235 !. % TODO: remove? - but proto_meta might have left choicepoints if loaded twice 236 237verify_version :- 238 ( protoc_gen_swipl_version(Module, Version), 239 Version @< [0,9,1] % This must be sync-ed with changes to protoc-gen-swipl 240 -> throw(error(version_error(Module-Version), _)) 241 ; true 242 ).
MessageType).
Protoc must have been run (with the --swipl_out= option and the resulting
top-level _pb.pl file loaded. For more details, see the "protoc" section of the
overview documentation.
Fails if the term isn't of an appropriate form or if the appropriate
meta-data from protoc hasn't been loaded, or if a field name is incorrect
(and therefore nothing in the meta-data matches it).
277protobuf_serialize_to_codes(Term, MessageType0, WireCodes) :- 278 verify_version, 279 must_be(ground, MessageType0), 280 ( proto_meta_normalize(MessageType0, MessageType) 281 -> true 282 ; existence_error(protobuf_package, MessageType0) 283 ), 284 term_to_segments(Term, MessageType, Segments), 285 !, % TODO: remove 286 protobuf_segment_message(Segments, WireCodes), 287 !. % TODO: remove? - but proto_meta might have left choicepoints if loaded twice 288 289% 290% Map wire type (atom) to its encoding (an int) 291% 292wire_type(varint, 0). % for int32, int64, uint32, uint64, sint32, sint64, bool, enum 293wire_type(fixed64, 1). % for fixed64, sfixed64, double 294wire_type(length_delimited, 2). % for string, bytes, embedded messages, packed repeated fields 295wire_type(start_group, 3). % for groups (deprecated) 296wire_type(end_group, 4). % for groups (deprecated) 297wire_type(fixed32, 5). % for fixed32, sfixed32, float 298 299% 300% basic wire-type processing handled by C-support code in DCG-form 301% 302 303fixed_uint32(X, [A0, A1, A2, A3 | Rest], Rest) :- 304 uint32_codes_when(X, [A0, A1, A2, A3]). 305/* equivalent to: 306fixed_uint32_(X) --> 307 [ A0,A1,A2,A3 ], 308 { uint32_codes_when(X, [A0,A1,A2,A3]) }. 309*/ 310 311fixed_uint64(X, [A0, A1, A2, A3, A4, A5, A6, A7 | Rest], Rest) :- 312 uint64_codes_when(X, [A0, A1, A2, A3, A4, A5, A6, A7]). 313 314fixed_float64(X, [A0, A1, A2, A3, A4, A5, A6, A7 | Rest], Rest) :- 315 float64_codes_when(X, [A0, A1, A2, A3, A4, A5, A6, A7]). 316 317fixed_float32(X, [A0, A1, A2, A3 | Rest], Rest) :- 318 float32_codes_when(X, [A0, A1, A2, A3]). 319 320% 321% Start of the DCG 322% 323 324code_string(N, Codes, Rest, Rest1) :- 325 length(Codes, N), 326 append(Codes, Rest1, Rest), 327 !. 328/* 329code_string(N, Codes) --> 330 { length(Codes, N) }, 331 Codes, !. 332*/ 333 334% 335% deal with Google's method of packing unsigned integers in variable 336% length, modulo 128 strings. 337% 338% protobuf_var_int//1 and protobuf_tag_type//2 productions were rewritten in straight 339% Prolog for speed's sake. 340%
A is negative.
This is a low-level predicate; normally, you should use
template_message/2 and the appropriate template term.
e.g. phrase(protobuf_var_int(300), S) => S = [172,2]
phrase(protobuf_var_int(A), [172,2]) -> A = 300350protobuf_var_int(A, [A | Rest], Rest) :- 351 A < 128, 352 !. 353protobuf_var_int(X, [A | Rest], Rest1) :- 354 nonvar(X), 355 X1 is X >> 7, 356 A is 128 + (X /\ 0x7f), 357 protobuf_var_int(X1, Rest, Rest1), 358 !. 359protobuf_var_int(X, [A | Rest], Rest1) :- 360 protobuf_var_int(X1, Rest, Rest1), 361 X is (X1 << 7) + A - 128, 362 !.
371protobuf_tag_type(Tag, WireType, Rest, Rest1) :- 372 nonvar(Tag), nonvar(WireType), 373 wire_type(WireType, WireTypeEncoding), 374 A is Tag << 3 \/ WireTypeEncoding, 375 protobuf_var_int(A, Rest, Rest1), 376 !. 377protobuf_tag_type(Tag, WireType, Rest, Rest1) :- 378 protobuf_var_int(A, Rest, Rest1), 379 WireTypeEncoding is A /\ 0x07, 380 wire_type(WireType, WireTypeEncoding), 381 Tag is A >> 3.
389prolog_type(Tag, double) --> protobuf_tag_type(Tag, fixed64). 390prolog_type(Tag, integer64) --> protobuf_tag_type(Tag, fixed64). 391prolog_type(Tag, unsigned64) --> protobuf_tag_type(Tag, fixed64). 392prolog_type(Tag, float) --> protobuf_tag_type(Tag, fixed32). 393prolog_type(Tag, integer32) --> protobuf_tag_type(Tag, fixed32). 394prolog_type(Tag, unsigned32) --> protobuf_tag_type(Tag, fixed32). 395prolog_type(Tag, integer) --> protobuf_tag_type(Tag, varint). 396prolog_type(Tag, unsigned) --> protobuf_tag_type(Tag, varint). 397prolog_type(Tag, signed32) --> protobuf_tag_type(Tag, varint). 398prolog_type(Tag, signed64) --> protobuf_tag_type(Tag, varint). 399prolog_type(Tag, boolean) --> protobuf_tag_type(Tag, varint). 400prolog_type(Tag, enum) --> protobuf_tag_type(Tag, varint). 401prolog_type(Tag, atom) --> protobuf_tag_type(Tag, length_delimited). 402prolog_type(Tag, codes) --> protobuf_tag_type(Tag, length_delimited). 403prolog_type(Tag, utf8_codes) --> protobuf_tag_type(Tag, length_delimited). 404prolog_type(Tag, string) --> protobuf_tag_type(Tag, length_delimited). 405prolog_type(Tag, embedded) --> protobuf_tag_type(Tag, length_delimited). 406prolog_type(Tag, packed) --> protobuf_tag_type(Tag, length_delimited). 407 408% 409% The protobuf-2.1.0 grammar allows negative values in enums. 410% But they are encoded as unsigned in the golden message. 411% As such, they use the maximum length of a varint, so it is 412% recommended that they be non-negative. However, that's controlled 413% by the =|.proto|= file. 414% 415:- meta_predicate enumeration(,,). 416 417enumeration(Type) --> 418 { call(Type, Value) }, 419 payload(signed64, Value).
Payload, according to PrologType
TODO: payload//2 "mode" is sometimes module-sensitive, sometimes not.
payload(enum, A)// has A as a callable
all other uses of payload//2, the 2nd arg is not callable.
vector_demo.pl, which defines commands/2)429payload(enum, Payload) --> 430 enumeration(Payload). 431payload(double, Payload) --> 432 fixed_float64(Payload). 433payload(integer64, Payload) --> 434 { uint64_int64_when(Payload0, Payload) }, 435 fixed_uint64(Payload0). 436payload(unsigned64, Payload) --> 437 fixed_uint64(Payload). 438payload(float, Payload) --> 439 fixed_float32(Payload). 440payload(integer32, Payload) --> 441 { uint32_int32_when(Payload0, Payload) }, 442 fixed_uint32(Payload0). 443payload(unsigned32, Payload) --> 444 fixed_uint32(Payload). 445payload(integer, Payload) --> 446 { nonvar(Payload), int64_zigzag(Payload, X) }, % TODO: int64_zigzag_when/2 447 !, 448 protobuf_var_int(X). 449payload(integer, Payload) --> 450 protobuf_var_int(X), 451 { int64_zigzag(Payload, X) }. % TODO: int64_zigzag_when/2 452payload(unsigned, Payload) --> 453 protobuf_var_int(Payload), 454 { Payload >= 0 }. 455payload(signed32, Payload) --> % signed32 is not defined by prolog_type//2 456 % for wire-stream compatibility reasons. 457 % signed32 ought to write 5 bytes for negative numbers, but both 458 % the C++ and Python implementations write 10 bytes. For 459 % wire-stream compatibility, we follow C++ and Python, even though 460 % protoc decode appears to work just fine with 5 bytes -- 461 % presumably there are some issues with decoding 5 bytes and 462 % getting the sign extension correct with some 32/64-bit integer 463 % models. See CodedOutputStream::WriteVarint32SignExtended(int32 464 % value) in google/protobuf/io/coded_stream.h. 465 payload(signed64, Payload). 466payload(signed64, Payload) --> 467 % protobuf_var_int//1 cannot handle negative numbers (note that 468 % zig-zag encoding always results in a positive number), so 469 % compute the 64-bit 2s complement, which is what is produced 470 % form C++ and Python. 471 { nonvar(Payload) }, 472 !, 473 { uint64_int64(X, Payload) }, % TODO: uint64_int64_when 474 protobuf_var_int(X). 475payload(signed64, Payload) --> 476 % See comment in previous clause about negative numbers. 477 protobuf_var_int(X), 478 { uint64_int64(X, Payload) }. % TODO: uint64_int64_when 479payload(codes, Payload) --> 480 { nonvar(Payload), 481 !, 482 length(Payload, Len) 483 }, 484 protobuf_var_int(Len), 485 code_string(Len, Payload). 486payload(codes, Payload) --> 487 protobuf_var_int(Len), 488 code_string(Len, Payload). 489payload(utf8_codes, Payload) --> 490 { nonvar(Payload), % TODO: use freeze/2 or when/2 491 !, 492 phrase(utf8_codes(Payload), B) 493 }, 494 payload(codes, B). 495payload(utf8_codes, Payload) --> 496 payload(codes, B), 497 { phrase(utf8_codes(Payload), B) }. 498payload(atom, Payload) --> 499 { nonvar(Payload), 500 atom_codes(Payload, Codes) 501 }, 502 payload(utf8_codes, Codes), 503 !. 504payload(atom, Payload) --> 505 payload(utf8_codes, Codes), 506 { atom_codes(Payload, Codes) }. 507payload(boolean, true) --> 508 payload(unsigned, 1). 509payload(boolean, false) --> 510 payload(unsigned, 0). 511payload(string, Payload) --> 512 { nonvar(Payload) 513 -> string_codes(Payload, Codes) 514 ; true 515 }, 516 % string_codes produces a list of unicode, not bytes 517 payload(utf8_codes, Codes), 518 { string_codes(Payload, Codes) }. 519payload(embedded, protobuf(PayloadSeq)) --> 520 { ground(PayloadSeq), 521 phrase(protobuf(PayloadSeq), Codes) 522 }, 523 payload(codes, Codes), 524 !. 525payload(embedded, protobuf(PayloadSeq)) --> 526 payload(codes, Codes), 527 { phrase(protobuf(PayloadSeq), Codes) }. 528payload(packed, TypedPayloadSeq) --> 529 { TypedPayloadSeq =.. [PrologType, PayloadSeq], % TypedPayloadSeq = PrologType(PayloadSeq) 530 ground(PayloadSeq), 531 phrase(packed_payload(PrologType, PayloadSeq), Codes) 532 }, 533 payload(codes, Codes), 534 !. 535payload(packed, enum(EnumSeq)) --> 536 !, 537 % TODO: combine with next clause 538 % TODO: replace =.. with a predicate that gives all the possibilities - see detag/6. 539 { EnumSeq =.. [ Enum, Values ] }, % EnumSeq = Enum(Values) 540 payload(codes, Codes), 541 { phrase(packed_enum(Enum, Values), Codes) }. 542payload(packed, TypedPayloadSeq) --> 543 payload(codes, Codes), 544 % TODO: replace =.. with a predicate that gives all the possibilities - see detag/6. 545 { TypedPayloadSeq =.. [PrologType, PayloadSeq] }, % TypedPayloadSeq = PrologType(PayloadSeq) 546 { phrase(packed_payload(PrologType, PayloadSeq), Codes) }. 547 548packed_payload(enum, EnumSeq) --> 549 { ground(EnumSeq) }, !, 550 { EnumSeq =.. [EnumType, Values] }, % EnumSeq = EnumType(Values) 551 packed_enum(EnumType, Values). 552packed_payload(PrologType, PayloadSeq) --> 553 sequence_payload(PrologType, PayloadSeq). 554 555% sequence_payload//2 (because sequence//2 isn't compile-time expanded) 556sequence_payload(PrologType, PayloadSeq) --> 557 sequence_payload_(PayloadSeq, PrologType). 558 559sequence_payload_([], _PrologType) --> [ ]. 560sequence_payload_([Payload|PayloadSeq], PrologType) --> 561 payload(PrologType, Payload), 562 sequence_payload_(PayloadSeq, PrologType). 563 564packed_enum(Enum, [ A | As ]) --> 565 % TODO: replace =.. with a predicate that gives all the possibilities - see detag/6. 566 { E =.. [Enum, A] }, 567 payload(enum, E), 568 packed_enum(Enum, As). 569packed_enum(_, []) --> [ ]. 570 571start_group(Tag) --> protobuf_tag_type(Tag, start_group). 572 573end_group(Tag) --> protobuf_tag_type(Tag, end_group). 574% 575% 576nothing([]) --> [], !. 577 578protobuf([Field | Fields]) --> 579 % TODO: don't use =.. -- move logic to single_message 580 ( { Field = repeated_embedded(Tag, protobuf(EmbeddedFields), Items) } 581 -> repeated_embedded_messages(Tag, EmbeddedFields, Items) 582 ; { Field =.. [ PrologType, Tag, Payload] }, % Field = PrologType(Tag, Payload) 583 single_message(PrologType, Tag, Payload), 584 ( protobuf(Fields) 585 ; nothing(Fields) 586 ) 587 ), 588 !. 589 590repeated_message(repeated_enum, Tag, Type, [A | B]) --> 591 % TODO: replace =.. with a predicate that gives all the possibilities - see detag/6. 592 { TypedPayload =.. [Type, A] }, % TypedPayload = Type(A) 593 single_message(enum, Tag, TypedPayload), 594 ( repeated_message(repeated_enum, Tag, Type, B) 595 ; nothing(B) 596 ). 597repeated_message(Type, Tag, [A | B]) --> 598 { Type \= repeated_enum }, 599 single_message(Type, Tag, A), 600 repeated_message(Type, Tag, B). 601repeated_message(_Type, _Tag, A) --> 602 nothing(A). 603 604repeated_embedded_messages(Tag, EmbeddedFields, [protobuf(A) | B]) --> 605 { copy_term(EmbeddedFields, A) }, 606 single_message(embedded, Tag, protobuf(A)), !, 607 repeated_embedded_messages(Tag, EmbeddedFields, B). 608repeated_embedded_messages(_Tag, _EmbeddedFields, []) --> 609 [ ].
protobuf([...]).
The PrologType, Tag, Payload are from Field =.. [PrologType, Tag, Payload]
in the caller615single_message(repeated, Tag, enum(EnumSeq)) --> 616 !, 617 { EnumSeq =.. [EnumType, Values] }, % EnumSeq = EnumType(Values) 618 repeated_message(repeated_enum, Tag, EnumType, Values). 619single_message(repeated, Tag, Payload) --> 620 !, 621 % TODO: replace =.. with a predicate that gives all the possibilities - see detag/6. 622 { Payload =.. [PrologType, A] }, % Payload = PrologType(A) 623 { PrologType \= enum }, 624 repeated_message(PrologType, Tag, A). 625single_message(group, Tag, A) --> 626 !, 627 start_group(Tag), 628 protobuf(A), 629 end_group(Tag). 630single_message(PrologType, Tag, Payload) --> 631 { PrologType \= repeated, PrologType \= group }, 632 prolog_type(Tag, PrologType), 633 payload(PrologType, Payload).
658protobuf_message(protobuf(TemplateList), WireStream) :- 659 must_be(list, TemplateList), 660 phrase(protobuf(TemplateList), WireStream), 661 !. 662 663protobuf_message(protobuf(TemplateList), WireStream, Residue) :- 664 must_be(list, TemplateList), 665 phrase(protobuf(TemplateList), WireStream, Residue).
.proto description, similar to
the processing done by protoc --decode_raw. This means that
field names aren't shown, only field numbers.
For unmarshalling, a simple heuristic is used on length-delimited segments: first interpret it as a message; if that fails, try to interpret as a UTF8 string; otherwise, leave it as a "blob" (if the heuristic was wrong, you can convert to a string or a blob by using protobuf_segment_convert/2). 32-bit and 64-bit numbers are left as codes because they could be either integers or floating point (use int32_codes_when/2, float32_codes_when/2, int64_codes_when/2, uint32_codes_when/2, uint64_codes_when/2, float64_codes_when/2 as appropriate); variable-length numbers ("varint" in the Protocol Buffers encoding documentation), might require "zigzag" conversion, int64_zigzag_when/2.
For marshalling, use the predicates int32_codes_when/2, float32_codes_when/2, int64_codes_when/2, uint32_codes_when/2, uint64_codes_when/2, float64_codes_when/2, int64_zigzag_when/2 to put integer and floating point values into the appropriate form.
738protobuf_segment_message(Segments, WireStream) :- 739 phrase(segment_message(Segments), WireStream). 740 741segment_message(Segments) --> 742 sequence_segment(Segments). 743 744% sequence_segment//1 (because sequence//2 isn't compile-time expanded) 745sequence_segment([]) --> [ ]. 746sequence_segment([Segment|Segments]) --> 747 segment(Segment), 748 sequence_segment(Segments). 749 750segment(Segment) --> 751 { nonvar(Segment) }, 752 !, 753 % repeated(List) can be created by field_segment_scalar_or_repeated/7 754 ( { Segment = repeated(Segments) } 755 -> sequence_segment(Segments) 756 ; { segment_type_tag(Segment, Type, Tag) }, 757 protobuf_tag_type(Tag, Type), 758 segment(Type, Tag, Segment) 759 ). 760segment(Segment) --> 761 % { var(Segment) }, 762 protobuf_tag_type(Tag, Type), 763 segment(Type, Tag, Segment). 764 765segment(varint, Tag, varint(Tag,Value)) --> 766 protobuf_var_int(Value). 767segment(fixed64, Tag, fixed64(Tag, Int64)) --> 768 payload(integer64, Int64). 769segment(fixed32, Tag, fixed32(Tag, Int32)) --> 770 payload(integer32, Int32). 771segment(start_group, Tag, group(Tag, Segments)) --> 772 segment_message(Segments), 773 protobuf_tag_type(Tag, end_group). 774segment(length_delimited, Tag, Result) --> 775 segment_length_delimited(Tag, Result). 776 777segment_length_delimited(Tag, Result) --> 778 { nonvar(Result) }, 779 !, 780 { length_delimited_segment(Result, Tag, Codes) }, 781 { length(Codes, CodesLen) }, 782 protobuf_var_int(CodesLen), 783 code_string(CodesLen, Codes). 784segment_length_delimited(Tag, Result) --> 785 % { var(Result) }, 786 protobuf_var_int(CodesLen), 787 code_string(CodesLen, Codes), 788 { length_delimited_segment(Result, Tag, Codes) }. 789 790length_delimited_segment(message(Tag,Segments), Tag, Codes) :- 791 protobuf_segment_message(Segments, Codes). 792length_delimited_segment(group(Tag,Segments), Tag, Codes) :- 793 phrase(segment_group(Tag, Segments), Codes). 794length_delimited_segment(string(Tag,String), Tag, Codes) :- 795 ( nonvar(String) 796 -> string_codes(String, StringCodes), 797 phrase(utf8_codes(StringCodes), Codes) 798 ; phrase(utf8_codes(StringCodes), Codes), 799 string_codes(String, StringCodes) 800 ). 801length_delimited_segment(packed(Tag,Payload), Tag, Codes) :- 802 % We don't know the type of the fields, so we try the 3 803 % possibilities. This has a problem: an even number of fixed32 804 % items can't be distinguished from half the number of fixed64 805 % items; but it's all we can do. The good news is that usually 806 % varint (possibly with zig-zag encoding) is more common because 807 % it's more compact (I don't know whether 32-bit or 64-bit is more 808 % common for floating point). 809 packed_option(Type, Items, Payload), 810 phrase(sequence_payload(Type, Items), Codes). 811length_delimited_segment(length_delimited(Tag,Codes), Tag, Codes). 812 813segment_group(Tag, Segments) --> 814 start_group(Tag), 815 segment_message(Segments), 816 end_group(Tag). 817 818% See also prolog_type//2. Note that this doesn't handle repeated(List), 819% which is used internally (see field_segment_scalar_or_repeated/7). 820segment_type_tag(varint(Tag,_Value), varint, Tag). 821segment_type_tag(fixed64(Tag,_Value), fixed64, Tag). 822segment_type_tag(group(Tag,_Segments), start_group, Tag). 823segment_type_tag(fixed32(Tag,_Value), fixed32, Tag). 824segment_type_tag(length_delimited(Tag,_Codes), length_delimited, Tag). 825segment_type_tag(message(Tag,_Segments), length_delimited, Tag). 826segment_type_tag(packed(Tag,_Payload), length_delimited, Tag). 827segment_type_tag(string(Tag,_String), length_delimited, Tag).
Compound or the form Name(Tag,Value) and create a
new CompoundWithList that replaces Value with List. This is
used by packed_list/2 to transform [varint(1,0),varint(1,1)] to
varint(1,[0,1]).
Some of Compound items are impossible for packed with the
current protobuf spec, but they don't do any harm.
837detag(varint(Tag,Value), varint, Tag, Value, List, varint(List)). 838detag(fixed64(Tag,Value), fixed64, Tag, Value, List, fixed64(List)). 839detag(fixed32(Tag,Value), fixed32, Tag, Value, List, fixed32(List)). 840detag(length_delimited(Tag,Codes), length_delimited, Tag, Codes, List, length_delimited(List)). 841detag(message(Tag,Segments), message, Tag, Segments, List, message(List)). 842detag(packed(Tag,Payload), packed, Tag, Payload, List, packed(List)). % TODO: delete? 843detag(string(Tag,String), string, Tag, String, List, string(List)). 844 845% See also prolog_type//2, but pick only one for each wirestream type 846% For varint(Items), use one that doesn't do zigzag 847packed_option(integer64, Items, fixed64(Items)). 848packed_option(integer32, Items, fixed32(Items)). 849packed_option(unsigned, Items, varint(Items)). 850% packed_option(integer, Items, varint(Items)). 851% packed_option(double, Items, fixed64(Items)). 852% packed_option(float, Items, fixed32(Items)). 853% packed_option(signed64, Items, varint(Items)). 854% packed_option(boolean, Items, varint(Items)). 855% packed_option(enum, Items, varint(Items)).
Form1 is converted back to the original wire stream, then the
predicate non-deterimisticly attempts to convert the wire stream to
a string or length_delimited term (or both: the lattter
always succeeds).
The possible conversions are:
message(Tag,Segments) => string(Tag,String)
message(Tag,Segments) => length_delimited(Tag,Codes)
string(Tag,String) => length_delimited(Tag,Codes)
length_delimited(Tag,Codes) => length_delimited(Tag,Codes)
Note that for fixed32, fixed64, only the signed integer forms are given; if you want the floating point forms, then you need to do use int64_float64_when/2 and int32_float32_when/2.
For example:
~~~{.pl}
?- protobuf_segment_convert(
message(10,[fixed64(13,7309475598860382318)]),
string(10,"inputType")).
?- protobuf_segment_convert(
message(10,[fixed64(13,7309475598860382318)]),
length_delimited(10,[105,110,112,117,116,84,121,112,101])).
?- protobuf_segment_convert(
string(10, "inputType"),
length_delimited(10,[105,110,112,117,116,84,121,112,101])).
?- forall(protobuf_segment_convert(string(1999,"\x1\\x0\\x0\\x0\\x2\\x0\\x0\\x0\"),Z), writeln(Z)).
string(1999,
890 ) 891% packed(1999,fixed64([8589934593])) 892% packed(1999,fixed32([1,2])) 893% packed(1999,varint([1,0,0,0,2,0,0,0])) 894% length_delimited(1999,[1,0,0,0,2,0,0,0]) 895% ~~~ 896% These come from: 897% ~~~{.pl} 898% Codes = [82,9,105,110,112,117,116,84,121,112,101], 899% protobuf_message(protobuf([embedded(T1, protobuf([integer64(T2, I)]))]), Codes), 900% protobuf_message(protobuf([string(T,S)]), Codes). 901% T = 10, T1 = 10, T2 = 13, 902% I = 7309475598860382318, 903% S = "inputType". 904% ~~~ 905% 906% @bug This predicate is preliminary and may change as additional 907% functionality is added. 908% @bug This predicate will sometimes generate unexpected choice points, 909% Such as from =|protobuf_segment_convert(message(10,...), string(10,...))|= 910% 911% @param Form1 =|message(Tag,Pieces)|=, =|string(Tag,String)|=, =|length_delimited(Tag,Codes)|=, 912% =|varint(Tag,Value)|=, =|fixed64(Tag,Value)|=, =|fixed32(Tag,Value)|=. 913% @param Form2 similar to =Form1=. 914protobuf_segment_convert(Form, Form). % for efficiency, don't generate codes 915protobuf_segment_convert(Form1, Form2) :- 916 dif(Form1, Form2), % Form1=Form2 handled by first clause 917 protobuf_segment_message([Form1], WireCodes), 918 phrase(tag_and_codes(Tag, Codes), WireCodes), 919 length_delimited_segment(Form2, Tag, Codes). 920 921tag_and_codes(Tag, Codes) --> 922 protobuf_tag_type(Tag, length_delimited), 923 payload(codes, Codes). 924 925%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% 926% Documention of the foreign predicates, which are wrapped and exported.
This predicate delays until either Uint32 or Codes is
sufficiently instantiated.
There is also a non-delayed uint32_codes/2
SWI-Prolog doesn't have a 32-bit integer type, so 32-bit integer is simulated by doing a range check.
946uint32_codes_when(Uint32, Codes) :-
947 when((nonvar(Uint32) ; ground(Codes)), uint32_codes(Uint32, Codes)).
This predicate delays until either Int32 or Codes is
sufficiently instantiated.
There is also a non-delayed int32_codes/2
SWI-Prolog doesn't have a 32-bit integer type, so 32-bit integer is simulated by doing a range check.
967int32_codes_when(Int32, Codes) :- % TODO: unused
968 when((nonvar(Int32) ; ground(Codes)), int32_codes(Int32, Codes)).
This predicate delays until either Value or Codes is
sufficiently instantiated.
There is also a non-delayed float32_codes/2
982float32_codes_when(Value, Codes) :-
983 when((nonvar(Value) ; ground(Codes)), float32_codes(Value, Codes)).SWI-Prolog allows integer values greater than 64 bits, so a range check is done.
This predicate delays until either Uint64 or Codes is
sufficiently instantiated.
There is also a non-delayed uint64_codes/2
998% 999% @param Uint64 an unsigned integer 1000% @param Codes a list of 8 integers (codes) 1001% 1002% @error Type,Domain if =Uint64= or =Codes= are of the wrong 1003% type or out of range. 1004uint64_codes_when(Uint64, Codes) :- 1005 when((nonvar(Uint64) ; ground(Codes)), uint64_codes(Uint64, Codes)).
SWI-Prolog allows integer values greater than 64 bits, so a range check is done.
This predicate delays until either Int64 or Codes is
sufficiently instantiated.
There is also a non-delayed int64_codes/2
1020% 1021% @param Int64 an unsigned integer 1022% @param Codes a list of 8 integers (codes) 1023% 1024% @error Type,Domain if =Int64= or =Codes= are of the wrong 1025% type or out of range. 1026int64_codes_when(Int64, Codes) :- % TODO: unused 1027 when((nonvar(Int64) ; ground(Codes)), int64_codes(Int64, Codes)).
This predicate delays until either Value or Codes is
sufficiently instantiated.
There is also a non-delayed float64_codes/2
1046float64_codes_when(Value, Codes) :-
1047 when((nonvar(Value) ; ground(Codes)), float64_codes(Value, Codes)).sint32 and sint64 types. This is a
low-level predicate; normally, you should use template_message/2 and
the appropriate template term.
SWI-Prolog allows integer values greater than 64 bits, so a range check is done.
This predicate delays until either Original or Encoded is
sufficiently instantiated.
There is also a non-delayed int64_zigzag/2
1072int64_zigzag_when(Original, Encoded) :-
1073 when((nonvar(Original) ; nonvar(Encoded)), int64_zigzag(Original, Encoded)).uint64_int64(0xffffffffffffffff,-1).
This predicate delays until either Uint64 or Int64 is
sufficiently instantiated.
There is also a non-delayed uint64_int64/2
1091uint64_int64_when(Uint64, Int64) :- 1092 when((nonvar(Uint64) ; nonvar(Int64)), uint64_int64(Uint64, Int64)). 1093 1094% Reversed argument ordering for maplist/3 1095int64_uint64_when(Int64, Uint64) :- 1096 uint64_int64_when(Uint64, Int64).
This predicate delays until either Uint32 or Int32 is
sufficiently instantiated.
There is also a non-delayed uint32_int32/2
1113uint32_int32_when(Uint32, Int32) :- 1114 when((nonvar(Uint32) ; nonvar(Int32)), uint32_int32(Uint32, Int32)). 1115 1116% Reversed argument ordering for maplist/3 1117int32_uint32_when(Int32, Uint32) :- 1118 1119 uint32_int32_when(Uint32, Int32).
int64_float64(3ff0000000000000,1.0).
This predicate delays until either Int64 or Float64 is
sufficiently instantiated.
There is also a non-delayed uint64_int64/2
1137int64_float64_when(Int64, Float64) :-
1138 when((nonvar(Int64) ; nonvar(Float64)), int64_float64(Int64, Float64)).int32_float32(0x3f800000,1.0).
This predicate delays until either Int32 or Float32 is
sufficiently instantiated.
There is also a non-delayed uint32_int32/2
1156int32_float32_when(Int32, Float32) :- 1157 when((nonvar(Int32) ; nonvar(Float32)), int32_float32(Int32, Float32)). 1158 1159 1160%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% 1161% 1162% Use protobufs meta-data (see the section on protoc in the "overview" documentation). 1163 1164% The protoc plugin generates the following facts (all starting with "proto_meta_"). 1165% The are documented in protoc-gen-swipl and in the overview section. 1166 1167:- multifile 1168 proto_meta_normalize/2, % (Unnormalized, Normalized) 1169 proto_meta_package/3, % (Package, FileName, Options) 1170 proto_meta_message_type/3, % (Fqn, Package, Name) 1171 proto_meta_message_type_map_entry/1, % (Fqn) 1172 proto_meta_field_name/4, % (Fqn, FieldNumber, FieldName, FqnName) 1173 proto_meta_field_json_name/2, % (FqnName, JsonName) 1174 proto_meta_field_label/2, % (FqnName, LabelRepeatOptional) % LABEL_OPTIONAL, LABEL_REQUIRED, LABEL_REPEATED 1175 proto_meta_field_type/2, % (FqnName, Type) % TYPE_INT32, TYPE_MESSAGE, etc 1176 proto_meta_field_type_name/2, % (FqnName, TypeName) 1177 proto_meta_field_default_value/2, % (FqnName, DefaultValue) 1178 proto_meta_field_option_packed/1, % (FqnName) 1179 proto_meta_enum_type/3, % (FqnName, Fqn, Name) 1180 proto_meta_enum_value/3, % (FqnName, Name, Number) 1181 proto_meta_field_oneof_index/2, % (FqnName, Index) 1182 proto_meta_oneof/3. % (FqnName, Index, Name) 1183 1184proto_meta_enum_value_when(ContextType, EnumValue, IntValue) :- 1185 when((nonvar(EnumValue) ; nonvar(IntValue)), 1186 proto_meta_enum_value_(ContextType, EnumValue, IntValue)). 1187 1188proto_meta_enum_value_(ContextType, EnumValue, IntValue) :- 1189 ( proto_meta_enum_value(ContextType, EnumValue, IntValue) 1190 -> true 1191 ; existence_error(ContextType, EnumValue-IntValue) 1192 ). 1193 1194:- det(segment_to_term/3).
TODO: if performance is an issue, this code can be combined with
protobuf_segment_message/2 (and thereby avoid the use of protobuf_segment_convert/2)1200segment_to_term(ContextType0, Segment, FieldAndValue) => 1201 segment_type_tag(Segment, _, Tag), 1202 field_and_type(ContextType0, Tag, FieldName, _FqnName, ContextType, RepeatOptional, Type), 1203 ( RepeatOptional = repeat_packed 1204 -> convert_segment_packed(Type, ContextType, Tag, Segment, Value) 1205 ; convert_segment(Type, ContextType, Tag, Segment, Value) 1206 ), 1207 !, % TODO: get rid of this? 1208 FieldAndValue = field_and_value(FieldName,RepeatOptional,Value). 1209 1210% :- det(convert_segment_packed/5). % TODO: "succeeded with a choicepoint" 1211%! convert_segment_packed(+Type:atom, +ContextType:atom, +Tag:atom, ?Segment, ?Values) is det. 1212% Reversible on =Segment=, =Values=. 1213% 1214% TODO: these are very similar to convert_segment - can they be combined? 1215 1216convert_segment_packed('TYPE_DOUBLE', _ContextType, Tag, Segment0, Values) => 1217 freeze(Segment0, protobuf_segment_convert(Segment0, packed(Tag, fixed64(Values0)))), 1218 maplist(int64_float64_when, Values0, Values), !. 1219convert_segment_packed('TYPE_FLOAT', _ContextType, Tag, Segment0, Values) => 1220 freeze(Segment0, protobuf_segment_convert(Segment0, packed(Tag, fixed32(Values0)))), 1221 maplist(int32_float32_when, Values0, Values), !. 1222convert_segment_packed('TYPE_INT64', _ContextType, Tag, Segment0, Values) => 1223 freeze(Segment0, protobuf_segment_convert(Segment0, packed(Tag, varint(Values0)))), 1224 maplist(uint64_int64_when, Values0, Values). 1225convert_segment_packed('TYPE_UINT64', _ContextType, Tag, Segment0, Values) => 1226 protobuf_segment_convert(Segment0, packed(Tag, varint(Values))), !. 1227convert_segment_packed('TYPE_INT32', _ContextType, Tag, Segment0, Values) => 1228 freeze(Segment0, protobuf_segment_convert(Segment0, packed(Tag, varint(Values0)))), 1229 maplist(uint32_int32_when, Values0, Values). 1230convert_segment_packed('TYPE_FIXED64', _ContextType, Tag, Segment0, Values) => 1231 freeze(Segment0, protobuf_segment_convert(Segment0, packed(Tag, fixed64(Values0)))), 1232 maplist(int64_uint64_when, Values0, Values). 1233convert_segment_packed('TYPE_FIXED32', _ContextType, Tag, Segment0, Values) => 1234 freeze(Segment0, protobuf_segment_convert(Segment0, packed(Tag, fixed32(Values0)))), 1235 maplist(int32_uint32_when, Values0, Values). 1236convert_segment_packed('TYPE_BOOL', _ContextType, Tag, Segment0, Values) => 1237 freeze(Segment0, protobuf_segment_convert(Segment0, packed(Tag, varint(Values0)))), 1238 maplist(int_bool_when, Values0, Values). 1239% TYPE_STRING isn't allowed TODO: add it anyway? 1240% TYPE_GROUP isn't allowed 1241% TYPE_MESSAGE isn't allowed 1242% TYPE_BYTES isn't allowed TODO: add it anyway? 1243convert_segment_packed('TYPE_UINT32', _ContextType, Tag, Segment0, Values) => 1244 protobuf_segment_convert(Segment0, packed(Tag, varint(Values))), !. 1245convert_segment_packed('TYPE_ENUM', ContextType, Tag, Segment0, Values) => 1246 % uint64_int64_when(...), % TODO! https://github.com/SWI-Prolog/contrib-protobufs/issues/9 1247 freeze(Segment0, protobuf_segment_convert(Segment0, packed(Tag, varint(Values0)))), 1248 maplist(convert_enum(ContextType), Values, Values0). 1249convert_segment_packed('TYPE_SFIXED32', _ContextType, Tag, Segment0, Values) => 1250 protobuf_segment_convert(Segment0, packed(Tag, fixed32(Values))). 1251convert_segment_packed('TYPE_SFIXED64', _ContextType, Tag, Segment0, Values) => 1252 protobuf_segment_convert(Segment0, packed(Tag, fixed64(Values))). 1253convert_segment_packed('TYPE_SINT32', _ContextType, Tag, Segment0, Values) => 1254 freeze(Segment0, protobuf_segment_convert(Segment0, packed(Tag, varint(Values0)))), 1255 maplist(int64_zigzag_when, Values, Values0). 1256convert_segment_packed('TYPE_SINT64', _ContextType, Tag, Segment0, Values) => 1257 freeze(Segment0, protobuf_segment_convert(Segment0, packed(Tag, varint(Values0)))), 1258 maplist(int64_zigzag_when, Values, Values0). 1259% convert_segment_packed(Type, ContextType, Tag, Segment, Values) => % TODO: delete this clause 1260% domain_error(type(type=Type, % TODO: this is a bit funky 1261% context_type=ContextType), 1262% value(segment=Segment, 1263% tag=Tag, 1264% values=Values)). 1265 1266:- det(convert_segment/5).
Value from the combination of descriptor
"type" (in Type) and a Segment.
Reversible on Segment, Values.1271convert_segment('TYPE_DOUBLE', _ContextType, Tag, Segment0, Value) => 1272 Segment = fixed64(Tag,Int64), 1273 int64_float64_when(Int64, Value), 1274 protobuf_segment_convert(Segment0, Segment), !. 1275convert_segment('TYPE_FLOAT', _ContextType, Tag, Segment0, Value) => 1276 Segment = fixed32(Tag,Int32), 1277 int32_float32_when(Int32, Value), 1278 protobuf_segment_convert(Segment0, Segment), !. 1279convert_segment('TYPE_INT64', _ContextType, Tag, Segment0, Value) => 1280 Segment = varint(Tag,Value0), 1281 uint64_int64_when(Value0, Value), 1282 protobuf_segment_convert(Segment0, Segment), !. 1283convert_segment('TYPE_UINT64', _ContextType, Tag, Segment0, Value) => 1284 Segment = varint(Tag,Value), 1285 protobuf_segment_convert(Segment0, Segment), !. 1286convert_segment('TYPE_INT32', _ContextType, Tag, Segment0, Value) => 1287 Segment = varint(Tag,Value0), 1288 uint32_int32_when(Value0, Value), 1289 protobuf_segment_convert(Segment0, Segment), !. 1290convert_segment('TYPE_FIXED64', _ContextType, Tag, Segment0, Value) => 1291 Segment = fixed64(Tag,Value0), 1292 uint64_int64_when(Value, Value0), 1293 protobuf_segment_convert(Segment0, Segment), !. 1294convert_segment('TYPE_FIXED32', _ContextType, Tag, Segment0, Value) => 1295 Segment = fixed32(Tag,Value0), 1296 uint32_int32_when(Value, Value0), 1297 protobuf_segment_convert(Segment0, Segment), !. 1298convert_segment('TYPE_BOOL', _ContextType, Tag, Segment0, Value) => 1299 Segment = varint(Tag,Value0), 1300 int_bool_when(Value0, Value), 1301 protobuf_segment_convert(Segment0, Segment), !. 1302% convert_segment('TYPE_STRING', _ContextType, Tag, Segment0, Value) => 1303% Segment = string(Tag,ValueStr), 1304% protobuf_segment_convert(Segment0, Segment), !, 1305% ( false % TODO: control whether atom or string with an option 1306% -> atom_string(Value, ValueStr) 1307% ; Value = ValueStr 1308% ). 1309convert_segment('TYPE_STRING', _ContextType, Tag, Segment0, Value) => 1310 % TODO: option to control whether to use atom_string(Value,ValueStr) 1311 Segment = string(Tag,Value), 1312 protobuf_segment_convert(Segment0, Segment), !. 1313convert_segment('TYPE_GROUP', ContextType, Tag, Segment0, Value) => 1314 Segment = group(Tag,MsgSegments), 1315 % TODO: combine with TYPE_MESSAGE code: 1316 ( nonvar(Value) 1317 -> dict_pairs(Value, _, FieldValues), 1318 maplist(field_segment(ContextType), FieldValues, MsgSegments), 1319 protobuf_segment_convert(Segment0, Segment) 1320 ; protobuf_segment_convert(Segment0, Segment), 1321 maplist(segment_to_term(ContextType), MsgSegments, MsgFields), 1322 combine_fields(MsgFields, ContextType{}, Value) 1323 ), !. 1324convert_segment('TYPE_MESSAGE', ContextType, Tag, Segment0, Value) => 1325 Segment = message(Tag,MsgSegments), 1326 ( nonvar(Value) 1327 -> dict_pairs(Value, _, FieldValues), 1328 maplist(field_segment(ContextType), FieldValues, MsgSegments), 1329 protobuf_segment_convert(Segment0, Segment) 1330 ; protobuf_segment_convert(Segment0, Segment), 1331 maplist(segment_to_term(ContextType), MsgSegments, MsgFields), 1332 combine_fields(MsgFields, ContextType{}, Value) 1333 ), !. 1334convert_segment('TYPE_BYTES', _ContextType, Tag, Segment0, Value) => 1335 Segment = length_delimited(Tag,Value), 1336 protobuf_segment_convert(Segment0, Segment), !. 1337convert_segment('TYPE_UINT32', _ContextType, Tag, Segment0, Value) => 1338 Segment = varint(Tag,Value), 1339 protobuf_segment_convert(Segment0, Segment), !. 1340convert_segment('TYPE_ENUM', ContextType, Tag, Segment0, Value) => 1341 Segment = varint(Tag,Value0), 1342 convert_enum(ContextType, Value, Value0), % TODO: negative values: https://github.com/SWI-Prolog/contrib-protobufs/issues/9 1343 protobuf_segment_convert(Segment0, Segment), !. 1344convert_segment('TYPE_SFIXED32', _ContextType, Tag, Segment0, Value) => 1345 Segment = fixed32(Tag,Value), 1346 protobuf_segment_convert(Segment0, Segment), !. 1347convert_segment('TYPE_SFIXED64', _ContextType, Tag, Segment0, Value) => 1348 Segment = fixed64(Tag,Value), 1349 protobuf_segment_convert(Segment0, Segment), !. 1350convert_segment('TYPE_SINT32', _ContextType, Tag, Segment0, Value) => 1351 Segment = varint(Tag,Value0), 1352 int64_zigzag_when(Value, Value0), 1353 protobuf_segment_convert(Segment0, Segment), !. 1354convert_segment('TYPE_SINT64', _ContextType, Tag, Segment0, Value) => 1355 Segment = varint(Tag,Value0), 1356 int64_zigzag_when(Value, Value0), 1357 protobuf_segment_convert(Segment0, Segment), !. 1358 1359convert_enum(ContextType, Enum, Uint) :- 1360 uint64_int64_when(Uint, Int), 1361 proto_meta_enum_value_when(ContextType, Enum, Int). 1362 1363% TODO: use options to translate to/from false, true (see json_read/3) 1364int_bool(0, false). 1365int_bool(1, true). 1366 1367int_bool_when(Int, Bool) :- 1368 when((nonvar(Int) ; nonvar(Bool)), int_bool(Int, Bool)).
1371add_defaulted_fields(Value0, ContextType, Value) :-
1372 % Can use bagof or findall if we know that there aren't any
1373 % duplicated proto_meta_field_name/4 rules, although this isn't
1374 % strictly necessary (just avoids processing a field twice).
1375 ( setof(Name-DefaultValue, message_field_default(ContextType, Name, DefaultValue), DefaultValues)
1376 -> true
1377 ; DefaultValues = []
1378 ),
1379 foldl(add_empty_field_if_missing, DefaultValues, Value0, Value).1382message_field_default(ContextType, Name, DefaultValue) :- 1383 proto_meta_field_name(ContextType, _FieldNumber, Name, Fqn), 1384 proto_meta_field_default_value(Fqn, DefaultValue), 1385 % If the field is part of a "oneof" group, then there will be a 1386 % proto_meta_oneof entry for it (using the oneof_index). All 1387 % fields have a oneof_index, but our code doesn't depend on that. 1388 \+ (proto_meta_field_oneof_index(Fqn, OneofIndex), 1389 proto_meta_oneof(ContextType, OneofIndex, _)). 1390 1391add_empty_field_if_missing(FieldName-DefaultValue, Dict0, Dict) :- 1392 ( get_dict(FieldName, Dict0, _) 1393 -> Dict = Dict0 1394 ; put_dict(FieldName, Dict0, DefaultValue, Dict) 1395 ). 1396 1397:- det(combine_fields/3).
1407combine_fields([], MsgDict0, MsgDict) => 1408 is_dict(MsgDict0, ContextType), 1409 add_defaulted_fields(MsgDict0, ContextType, MsgDict). 1410combine_fields([field_and_value(Field,norepeat,Value)|Fields], MsgDict0, MsgDict) => 1411 put_dict(Field, MsgDict0, Value, MsgDict1), 1412 combine_fields(Fields, MsgDict1, MsgDict). 1413combine_fields([field_and_value(Field,repeat_packed,Values0)|Fields], MsgDict0, MsgDict) => 1414 ( get_dict(Field, MsgDict0, ExistingValues) 1415 -> append(ExistingValues, Values0, Values) 1416 ; Values = Values0 1417 ), 1418 put_dict(Field, MsgDict0, Values, MsgDict1), 1419 combine_fields(Fields, MsgDict1, MsgDict). 1420combine_fields([field_and_value(Field,repeat,Value)|Fields], MsgDict0, MsgDict) => 1421 combine_fields_repeat(Fields, Field, NewValues, RestFields), 1422 ( get_dict(Field, MsgDict0, ExistingValues) 1423 -> append(ExistingValues, [Value|NewValues], Values) 1424 ; Values = [Value|NewValues] 1425 ), 1426 put_dict(Field, MsgDict0, Values, MsgDict1), 1427 combine_fields(RestFields, MsgDict1, MsgDict). 1428 1429:- det(combine_fields_repeat/4).
Field - the assumption
is that all the items for a field will be together and if they're
not, they would be combined outside this predicate.
1440combine_fields_repeat([], _Field, Values, RestFields) => Values = [], RestFields = []. 1441combine_fields_repeat([Field-repeat-Value|Fields], Field, Values, RestFields) => 1442 Values = [Value|Values2], 1443 combine_fields_repeat(Fields, Field, Values2, RestFields). 1444combine_fields_repeat(Fields, _Field, Values, RestFields) => Values = [], RestFields = Fields. 1445 1446:- det(field_and_type/7).
ContextType and Tag to get the field name, type, etc.
1449field_and_type(ContextType, Tag, FieldName, FqnName, ContextType2, RepeatOptional, Type) =>
1450 assertion(ground(ContextType)), % TODO: remove
1451 assertion(ground(Tag)), % TODO: remove
1452 ( proto_meta_field_name(ContextType, Tag, FieldName, FqnName),
1453 proto_meta_field_type_name(FqnName, ContextType2),
1454 fqn_repeat_optional(FqnName, RepeatOptional),
1455 proto_meta_field_type(FqnName, Type)
1456 -> true % Remove choicepoint, if JITI didn't do the right thing.
1457 ; existence_error(ContextType, Tag)
1458 ).proto_meta_field_label(FqnName, _), proto_meta_field_option_packed(FqnName)
and set RepeatOptional to one of
norepeat, repeat, repeat_packed.1464fqn_repeat_optional(FqnName, RepeatOptional) => 1465 % TODO: existence_error if \+ proto_meta_field_label 1466 proto_meta_field_label(FqnName, LabelRepeatOptional), 1467 ( LabelRepeatOptional = 'LABEL_REPEATED', 1468 proto_meta_field_option_packed(FqnName) 1469 -> RepeatOptional = repeat_packed 1470 ; \+ proto_meta_field_option_packed(FqnName), % validity check 1471 fqn_repeat_optional_2(LabelRepeatOptional, RepeatOptional) 1472 ). 1473 1474:- det(fqn_repeat_optional_2/2).
proto_meta_enum_value('.google.protobuf.FieldDescriptorProto.Label', Label, _).1478fqn_repeat_optional_2('LABEL_OPTIONAL', norepeat). 1479fqn_repeat_optional_2('LABEL_REQUIRED', norepeat). 1480fqn_repeat_optional_2('LABEL_REPEATED', repeat).
proto_meta_enum_value('.google.protobuf.FieldDescriptorProto.Label', 'LABEL_REPEATED', _).
TODO: unused
1485field_descriptor_label_repeated('LABEL_REPEATED').proto_meta_enum_value('.google.protobuf.FieldDescriptorProto.Label', Label, _).1489field_descriptor_label_single('LABEL_OPTIONAL'). 1490field_descriptor_label_single('LABEL_REQUIRED'). 1491 1492:- det(term_to_segments/3).
Term, generating message segments1495term_to_segments(Term, MessageType, Segments) :- 1496 dict_pairs(Term, _, FieldValues), 1497 maplist(field_segment(MessageType), FieldValues, Segments). 1498 1499:- det(field_segment/3). 1500% MessageType is the FQN of the field type (e.g., '.test.Scalars1') 1501% FieldName-Value is from the dict_pairs of the term. 1502% TODO: Throw an error if proto_meta_field_name/4 fails (need to make 1503% sure of all the possible uses of field_segment/3 and that 1504% nothing depends on it being able to fail without an error). 1505field_segment(MessageType, FieldName-Value, Segment) :- 1506 ( proto_meta_field_name(MessageType, Tag, FieldName, FieldFqn), 1507 proto_meta_field_type(FieldFqn, FieldType), 1508 proto_meta_field_type_name(FieldFqn, FieldTypeName), 1509 proto_meta_field_label(FieldFqn, Label) 1510 -> true % Remove choicepoint, if JITI didn't do the right thing. 1511 ; existence_error(MessageType, FieldName-Value) 1512 ), 1513 ( proto_meta_field_option_packed(FieldFqn) 1514 -> Packed = packed 1515 ; Packed = not_packed 1516 ), 1517 field_segment_scalar_or_repeated(Label, Packed, FieldType, Tag, FieldTypeName, Value, Segment), 1518 !. % TODO: remove 1519 1520:- det(field_segment_scalar_or_repeated/7).
FieldType is from the .proto meta information ('TYPE_SINT32', etc.)1523field_segment_scalar_or_repeated('LABEL_OPTIONAL', not_packed, FieldType, Tag, FieldTypeName, Value, Segment) => 1524 convert_segment(FieldType, FieldTypeName, Tag, Segment, Value). 1525field_segment_scalar_or_repeated('LABEL_REQUIRED', not_packed, FieldType, Tag, FieldTypeName, Value, Segment) => % same as LABEL_OPTIONAL 1526 convert_segment(FieldType, FieldTypeName, Tag, Segment, Value). 1527field_segment_scalar_or_repeated('LABEL_REPEATED', packed, FieldType, Tag, FieldTypeName, Values, Segment) => 1528 Segment = packed(Tag,FieldValues), 1529 maplist(convert_segment_v_s(FieldType, FieldTypeName, Tag), Values, Segments0), 1530 packed_list(Segments0, FieldValues). 1531field_segment_scalar_or_repeated('LABEL_REPEATED', not_packed, FieldType, Tag, FieldTypeName, Values, Segment) => 1532 Segment = repeated(Segments), 1533 maplist(convert_segment_v_s(FieldType, FieldTypeName, Tag), Values, Segments). 1534% field_segment_scalar_or_repeated(Label, Packed, FieldType, Tag, FieldTypeName, Value, Segment) :- % TODO: delete this clause 1535% domain_error(type(field_type=FieldType, % TODO: this is a bit funky 1536% label=Label, 1537% packed=Packed), 1538% value(tag=Tag, field_type_name=FieldTypeName, value=Value, segment=Segment)). 1539 1540convert_segment_v_s(FieldType, FieldTypeName, Tag, Value, Segment) :- 1541 convert_segment(FieldType, FieldTypeName, Tag, Segment, Value). 1542 1543% Convert [varint(1,10),varint(1,20)] to varint(1,[10,20]). 1544packed_list([], []). 1545packed_list([T1|Ts], PackedList) :- 1546 detag(T1, Functor, Tag, _V1, List, PackedList), 1547 packed_list_([T1|Ts], Functor, Tag, List). 1548 1549% Functor and Tag are only for verifying that the terms are of the 1550% expected form. 1551packed_list_([], _, _, []). 1552packed_list_([T1|Ts], Functor, Tag, [X1|Xs]) :- 1553 detag(T1, Functor, Tag, X1, _, _), 1554 packed_list_(Ts, Functor, Tag, Xs).
MessageType's FieldName is defined as a map<...> in
the .proto file.
1559protobuf_field_is_map(MessageType0, FieldName) :-
1560 proto_meta_normalize(MessageType0, MessageType),
1561 proto_meta_field_name(MessageType, _, FieldName, FieldFqn),
1562 proto_meta_field_type(FieldFqn, 'TYPE_MESSAGE'),
1563 proto_meta_field_label(FieldFqn, 'LABEL_REPEATED'),
1564 proto_meta_field_type_name(FieldFqn, FieldTypeName),
1565 proto_meta_message_type_map_entry(FieldTypeName),
1566 assertion(proto_meta_field_name(FieldTypeName, 1, key, _)),
1567 assertion(proto_meta_field_name(FieldTypeName, 2, value, _)),
1568 !.DictTag{key:Key, value:Value} and a key-value list as described
in library(pairs). At least one of ProtobufTermList and Pairs
must be instantiated; DictTag can be uninstantiated. If
ProtobufTermList is from a term created by
protobuf_parse_from_codes/3, the ordering of the items is undefined;
you can order them by using keysort/2 (or by a predicate such as
dict_pairs/3, list_to_assoc/2, or list_to_rbtree/2.1579protobuf_map_pairs(ProtobufTermList, DictTag, Pairs) :- 1580 maplist(protobuf_dict_map_pairs(DictTag), ProtobufTermList, Pairs). 1581 1582protobuf_dict_map_pairs(DictTag, DictTag{key:Key,value:Value}, Key-Value)
Google's Protocol Buffers ("protobufs")
Protocol buffers are Google's language-neutral, platform-neutral, extensible mechanism for serializing structured data -- think XML, but smaller, faster, and simpler. You define how you want your data to be structured once. This takes the form of a template that describes the data structure. You use this template to encode and decode your data structure into wire-streams that may be sent-to or read-from your peers. The underlying wire stream is platform independent, lossless, and may be used to interwork with a variety of languages and systems regardless of word size or endianness. Techniques exist to safely extend your data structure without breaking deployed programs that are compiled against the "old" format.
The idea behind Google's Protocol Buffers is that you define your structured messages using a domain-specific language and tool set. Further documentation on this is at https://developers.google.com/protocol-buffers.
There are two ways you can use protobufs in Prolog:
.protofile: protobuf_parse_from_codes/3 and protobuf_serialize_to_codes/3.The protobuf_parse_from_codes/3 and protobuf_serialize_to_codes/3 interface translates between a "wire stream" and a Prolog term. This interface takes advantage of SWI-Prolog's dict. There is a
protocplugin (protoc-gen-swipl) that generates a Prolog file of meta-information that captures the.protofile's definition in theprotobufsmodule:proto_meta_normalize(Unnormalized, Normalized)proto_meta_package(Package, FileName, Options)proto_meta_message_type( Fqn, Package, Name)proto_meta_message_type_map_entry( Fqn)proto_meta_field_name( Fqn, FieldNumber, FieldName, FqnName)proto_meta_field_json_name( FqnName, JsonName)proto_meta_field_label( FqnName, LabelRepeatOptional) % 'LABEL_OPTIONAL', 'LABEL_REQUIRED', 'LABEL_REPEATED'proto_meta_field_type( FqnName, Type) % 'TYPE_INT32', 'TYPE_MESSAGE', etcproto_meta_field_type_name( FqnName, TypeName)proto_meta_field_default_value( FqnName, DefaultValue)proto_meta_field_option_packed( FqnName)proto_meta_enum_type( FqnName, Fqn, Name)proto_meta_enum_value( FqnName, Name, Number)proto_meta_field_oneof_index( FqnName, Index)proto_meta_oneof( FqnName, Index, Name)The protobuf_message/2 interface allows you to define your message template as a list of predefined Prolog terms that correspond to production rules in the Definite Clause Grammar (DCG) that realizes the interpreter. Each production rule has an equivalent rule in the protobuf grammar. The process is not unlike specifiying the format of a regular expression. To encode a template to a wire-stream, you pass a grounded template,
X, and variable,Y, to protobuf_message/2. To decode a wire-stream,Y, you pass an ungrounded template,X, along with a grounded wire-stream,Y, to protobuf_message/2. The interpreter will unify the unbound variables in the template with values decoded from the wire-stream.For an overview and tutorial with examples, see
library(protobufs): Google's Protocol Buffers Examples of usage may also be found by inspecting test_protobufs.pl and the demo directory, or by looking at the "addressbook" example that is typically installed at /usr/lib/swi-prolog/doc/packages/examples/protobufs/interop/addressbook.pl